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1.  Introduction 

This  report  describes  the  work  done  for  the  Starfire  Optical  Range,  Kirtland  Air  Force 
Base  under  Contract  N00014-01-D-043  DO  #11,  between  02  September  2004  and  30 
April  2006.  This  work  relates  to  the  Air  Force’s  need  to  characterize  the  cloud 
distribution  during  day  and  night,  for  a  variety  of  applications,  including  support  of 
research  into  impact  of  clouds  on  laser  communication  and  support  of  satellite  tracking. 
This  contract  followed  Contract  N00014-01-D-0043  DO  #4,  which  will  be  discussed  in 
Section  2,  and  is  documented  in  Shields  et  al  2007,  Technical  Note  271.  Under  this 
contract,  we  began  preparing  Whole  Sky  Imager  systems  for  field  experiments  in  support 
of  program  goals,  adapting  the  software  and  refurbishing  the  hardware.  Significant 
progress  was  made  both  in  the  related  cloud  algorithms  and  in  methods  to  assess  their 
accuracy. 

A  related  contract  was  funded  through  Boeing  during  31  January  2005  -  30  November 
2005.  The  tasks  completed  under  that  contract  are  closely  related  to  these  tasks,  and  will 
also  be  reported  here.  In  particular,  early  portions  of  the  night  algorithm  work  reported  in 
Section  7,  and  early  portions  of  the  hardware  and  software  refurbishment  were  completed 
partly  under  the  ONR  contract  and  partly  under  the  Boeing  contract.  The  work  under  this 
Boeing  contract  was  finished  in  May  2005. 

A  follow-on  contract,  ONR  N00014-01-D-0043  DO  #13  was  funded  on  20  April  2006. 
The  work  under  DO  #13  will  be  reported  under  a  separate  report  upon  completion  of  the 
contract. 

2.  Background 

A  series  of  digital,  automated  Whole  Sky  Imagers  (WSI)  have  been  developed  by  MPL 
over  many  years,  beginning  in  the  early  1980’s  (Johnson  et  al.  1989  and  1991,  and 
Shields  et  al.  1993,  1994,  1997a  and  b).  (Published  references  are  listed  in  section  12.3.) 
These  systems  are  designed  to  acquire  accurate  imagery  of  the  full  upper  hemisphere  in 
several  spectral  filters,  in  order  to  assess  the  presence  of  clouds  at  each  pixel  in  the 
image.  A  system  capable  of  24-hour  operation,  the  Day/Night  WSI,  was  developed  by 
MPL  under  funding  from  the  Air  Force,  Navy,  and  Army  in  the  early  1990’s  (Shields  et 
al.  1998,  2003b  and  e,  2004  a  and  b,  and  2005b  and  c).  One  of  the  first  two  units  was 
fielded  at  the  Air  Force’s  Starfire  Optical  Range  in  October  1992. 
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Related  systems  have  been  designed  and  fielded  over  the  years.  These  include  a  new 
Daytime  Visible/NIR  WSI  (Feister  et  al.  2000,  and  Shields  et  al.  2003d),  and  visible  and 
Short-wave  IR  systems  for  airborne  use  (Shields  et  al.  2003c).  An  imaging  system  for 
measuring  visibility  was  developed  and  successfully  tested  (Shields  et  al.  2005a  and 
2006).  Also,  a  field  calibration  device  for  use  with  the  Day/Night  WSI  was  developed 
(Shields  et  al.  2003a). 

Partly  as  a  result  of  SOR’s  experience  with  the  Day/Night  WSI  fielded  in  1992,  they  have 
funded  development  of  additional  instruments,  as  well  as  algorithm  development  and  data 
analysis  in  recent  years.  Under  Contract  N00014-97-D-0350  DO  #2  (September  1997  - 
June  2001),  MPL  was  funded  to  develop  and  provide  a  new  Day/Night  WSI,  which  was 
designated  Unit  12  (Shields  et  al.  2003b).  This  unit  included  several  design  upgrades 
developed  for  other  sponsors.  The  funding  also  included  analysis  and  processing  of 
existing  daytime  cloud  decision  images  to  provide  statistical  estimates  of  Cloud  Free  Line 
of  Sight  (CFLOS)  and  related  properties.  Under  the  optional  funding,  the  WSI  was  also 
upgraded  to  run  under  Windows.  This  was  a  major  upgrade,  involving  a  new  camera 
model,  a  new  camera  software  library,  and  new  WSI  instrument  control  software.  The 
instrument  was  delivered  in  January  1999,  and  has  been  running  well  for  much  of  the 
time  since.  The  data  analysis  results  were  also  delivered  in  1998  and  1999,  and  my 
understanding  is  that  these  data  have  proven  to  be  quite  useful. 

Under  Contract  N00014-97-D-0350  DO  #6  (May  1999  -  May  2003),  we  were  funded  to 
provide  two  additional  instruments,  Units  13  and  14  and  do  additional  analysis  work 
(Shields  et  al.  2004b).  These  instruments  included  significant  design  upgrades,  including 
integration  of  the  control  computer  into  the  outdoor  environmental  housing,  new  control 
hardware  and  software,  and  new  software  to  provide  near-real-time  cloud  processing  on 
an  additional  display  computer.  One  of  the  instruments  was  fielded  at  a  site  in  California 
for  an  experiment,  and  ran  well  prior  to  the  completion  of  the  experiment,  at  which  time 
it  was  returned  to  MPL.  The  other  system  was  kept  at  MPL  pending  sponsor  readiness 
for  deployment.  It  was  allowed  to  run  continuously,  and  ran  well. 

In  addition,  under  the  options,  we  were  funded  to  develop  a  night  cloud  algorithm  based 
on  detection  of  the  contrast  between  the  signal  from  stars  and  their  background.  The 
concepts  had  been  developed  under  funding  from  another  sponsor  (Shields  et  al.  2002), 
and  under  Delivery  Order  6,  this  concept  was  expanded  to  handle  moonlight,  and 
converted  to  a  fieldable  C-code.  The  appropriate  geometric  calibrations  and  background 
were  extracted  for  the  Unit  12  running  at  the  SOR  site,  and  the  algorithm  was  installed 
and  provided  reasonable  results. 

Under  Contract  N00014-01-D-0043  DO  #4  (May  2001  -  September  2006),  we  continued 
hardware,  algorithm,  and  analysis  work  (Shields  et  al.  2007).  We  fielded  WSI  Unit  14  at 
a  site  in  Virginia,  and  supported  this  deployment  as  well  as  the  continued  operation  of 
Unit  12  at  the  SOR  site.  Although  Unit  14  ran  flawlessly  on  the  earlier  deployment  and 
in  extensive  tests  at  MPL,  we  had  more  problems  than  normal  at  the  Virginia  site,  but 
were  able  to  keep  it  operational  much  of  the  time.  The  Unit  12  system  generally  operated 
well,  although  it  required  extensive  repair  following  a  lightning  strike,  and  somewhat 
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more  repairs  than  normal  during  the  period  after  that.  We  were  pleased  to  be  able  to 
return  it  to  operation  after  such  an  event.  We  also  began  evaluation  of  how  to  build  a 
new  WSI  with  currently  available  technology,  and  how  to  simplify  the  system  for 
increased  robustness  and  decreased  cost. 

Also  under  DO  #4  the  day  algorithm  at  the  SOR  site  was  updated  to  run  in  real  time,  and 
include  a  horizon  and  occultor  mask,  and  the  clear  sky  background  was  updated.  A 
stand-alone  version  of  the  algorithm  was  written,  and  under  funding  from  another 
sponsor,  an  extensive  data  base  was  processed  and  analyzed  to  extract  CFLOS  statistics. 
This  work  allowed  us  to  detennine  the  strengths  and  weaknesses  of  the  daytime  cloud 
algorithm.  The  night  algorithm  was  further  developed  by  detennining  appropriate 
upgrades  and  inputs  required  for  the  SOR  site,  which  was  more  impacted  by 
anthropogenic  light  sources  than  previous  sites.  The  new  night  algorithm  was  installed  to 
run  in  real  time  at  the  SOR  site.  This  was  the  first  time  that  we  had  a  night  algorithm 
installed  so  that  it  could  provide  processed  results  in  real  time. 

Toward  the  end  of  this  prior  contract,  a  funding  increment  was  received  that  partially 
funded  the  new  contract  (DO  #11),  and  partially  funded  the  existing  contract  (DO  #4). 
The  Statement  of  Work  (SOW)  for  the  new  contract,  DO  #11,  reflected  the  priorities  at 
the  time  of  the  funding  increment.  However,  we  were  asked  to  use  the  options  under  the 
existing  contract,  DO  #4,  to  begin  the  work.  As  a  result,  we  were  able  to  provide  an 
extensive  analysis  of  IR  systems  and  their  pros  and  cons  for  this  program.  Most  of  this 
analysis  concentrated  on  Long  Wave  IR  (LWIR)  systems  in  the  8-12  pm  wavelength 
region,  because  our  analysis  showed  that  Short  Wave  IR  (SWIR)  systems  near  the  1.6  pm 
wavelength  region  would  not  be  adequate  for  our  purposes,  and  Mid  Wave  IR  (MWIR) 
systems  near  the  3-5  pm  wavelength  region  had  disadvantages  with  respect  to  the 
LWIR  systems.  The  LWIR  analysis  of  theoretical  perfonnance  and  images  we  had 
access  to  showed  that  the  LWIR  system  probably  will  not  detect  most  clouds  near  the 
horizon  in  normal  haze  environments,  and  will  have  difficulty  detecting  high  clouds  over 
many  parts  of  the  sky  under  many  conditions.  This  analysis,  as  well  as  the  other  work 
done  under  DO  #4,  is  presented  in  Shields  et  al.  2007. 

3.  Statement  of  Work 

The  Statement  of  Work  for  the  Delivery  Order  #1 1  is  given  in  italics  below. 

Primary  Task: 

The  contractor  shall,  unless  otherwise  specified  herein,  supply  the  necessary  personnel, 
facilities,  services,  and  materials  to  accomplish  the  following  tasks  within  a  one  year 
period  following  receipt  of funding. 

1.  Upgrade  and  Evaluation  of  Current  WSI  Capabilities:  Evaluate  night  and  daytime 
cloud  decision  algorithms.  SOR  will  supply  WSI  raw  images  and  corresponding  lidar 
data  to  MPL  to  use  in  this  evaluation.  Implement  upgrades  to  stand-alone  versions  of 
the  cloud  algorithms.  Document  significant  findings  and  prevent  [present]  a  review 
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of  recommended  algorithm  improvements  for  use  in  the  field  to  achieve  accurate  24/7 
cloud  assessment. 

2.  Analysis  of  Infrared  (IR)  Sensors:  Analyze  sensor  performance  and  anticipated 
impacts  of  cloud  and  sky  background  flux  levels  for  IR  regions.  The  analysis  should 
result  in  an  estimate  of  whether  an  IR  sensor  would  improve  the  capability  of 
determining  cloud/no  cloud  whole  sky  decisions.  SOR  will  furnish  to  MPL 
specifications  of  IR  sensors  available  for  use,  although  MPL  is  not  restricted  to  using 
these  sensors.  This  effort  will  provide  a  comparison  of  the  expected  performance  of 
an  IR  system  in  comparison  to  the  performance  of  the  existing  visible  systems.  At  the 
end  of  this  task,  MPL  shall  document  significant  findings  and  make  recommendations 
on  IR  system  development. 

3.  Evaluation  of  System  Upgrades:  MPL  shall  conduct  an  evaluation  of  overall  system 
upgrades  to  make  the  existing  visible  unit  more  robust  and  to  simplify  hardware  and 
software. 

Optional  Tasks: 

4.  Coordinate  with  the  sponsor  regarding  the  most  appropriate  tasks  and  estimated 
costs  for  further  development.  Tasks  are  anticipated  to  include  one  or  more  of  the 
following: 

4a.  Provide  algorithm  or  hardware  support  of  the  existing  SOR  WSI  units. 

4b.  Begin  work  toward  designing  and  building  an  IR  WSI  unit. 

4c.  Begin  work  toward  building  multiple  Visible  WSIs  units. 

5.  Provide  personnel  trained  in  the  WSI  and  its  capabilities  to  address  these  tasks  to  the 
limit  of  funding  provided  under  the  optional  budget.  These  tasks  may  include 
analysis,  software  development,  documentation,  minor  hardware  development,  and 
other  tasks  related  to  the  WSI  and  are  mutually  agreed  upon  by  the  sponsor  and  by 
MPL  to  be  appropriate. 

4.  Funding  Increments  and  Optional  Tasks 

The  funding  was  sent  from  the  Air  Force  to  ONR  in  two  funding  increments,  and  the 
optional  tasks  were  defined  as  these  funding  increments  were  received.  An  initial 
funding  increment  was  sent  to  ONR  in  a  MIPR  dated  10  May  2004.  Some  of  these  funds 
were  allocated  into  the  previous  Contract  N00014-01-D-0043  DO  #4,  and  the  remaining 
funds  were  put  into  the  new  Contract  N00014-01-D-043  DO  #11.  The  new  contract  was 
received  at  MPL  in  September  2004.  This  funded  the  primary  task  budget,  and  the 
priority  was  to  work  on  the  tasks  listed  in  the  Statement  of  Work  Primary  Tasks  section. 
As  noted  in  Section  2,  we  were  able  to  complete  portions  of  the  second  item  in  the  SOW 
under  the  earlier  contract  with  this  first  split  funding  increment. 

During  this  time,  SOR  decided  to  field  3  sites  with  WSI’s,  in  support  of  a  broader 
program.  They  did  not  have  funding  to  have  us  build  and  deploy  new  WSI  units, 
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however  some  older  WSI  units  that  had  been  built  by  MPL  under  a  DOE  program 
became  available  to  MPL.  It  was  decided  that  these  should  be  refurbished  for  use  at 
these  test  sites,  and  refurbishment  was  begun  under  this  program.  This  included 
hardware  refurbishment  such  as  replacing  worn  or  broken  parts  and  integrating  a  second 
computer  for  real-time  processing.  Extensive  software  rewrite  was  also  required. 
Support  of  logistics  planning  for  deployment  was  included.  We  were  given  permission  to 
start  this  work,  as  part  of  Primary  Task  3. 

The  second  funding  increment  was  originally  issued  January  2004;  errors  in  it  were 
corrected  in  March  2004,  and  it  was  received  in  May  2004.  This  increment  funded  most 
of  the  optional  task  budget;  the  remaining  funding  was  never  sent.  Under  this  MIPR,  the 
priorities  were  to 

1.  Continue  with  the  WSI  system,  software,  and  algorithm  refurbishments,  to  within  the 
level  possible  within  this  funding. 

2.  Work  toward  assessing  the  accuracy  of  the  algorithms  and  their  application  for 
multiple  sites. 

3.  To  the  extent  feasible  under  funding,  support  the  maintenance  of  the  fielded  units, 
particularly  the  unit  at  the  SOR  site. 

These  three  items  may  be  considered  part  of  Optional  Task  4a  from  the  SOW.  Optional 
Task  4b,  which  related  to  building  an  IR  system,  was  not  funded,  because  the  IR  analysis 
indicated  that  IR  systems  were  unlikely  to  do  a  better  job  at  meeting  project  cloud 
detection  needs.  Optional  Task  4c,  related  to  building  new  WSI  systems,  was  not  funded 
because  the  older  WSI  systems  became  available  for  refurbishment. 

5.  Major  Deliveries  and  Documentation 

These  tasks  are  somewhat  open  ended,  however  major  progress  was  completed  in  all 
areas,  as  required  to  meet  the  Statement  of  Work.  We  would  like  to  document  the  major 
deliveries  and  documentation  for  these  tasks.  In  the  later  sections,  we  will  provide  an 
overview  of  the  more  significant  advances  related  to  general  capabilities.  Additional 
documentation  is  provided  in  the  technical  memoranda  listed  below  and  in  the  reference 
section  12.1.  These  technical  memos  can  be  provided  to  the  sponsor  upon  request.  In 
addition,  this  work  was  presented  to  sponsors  at  several  meetings.  The  available  Power 
Point  files  are  listed  in  the  References  section  12.1. 

5,1,  Algorithm  and  Ground-truthing  Developments 

Primary  Task  1:  This  task  is  directed  primarily  toward  evaluating  the  day  and  night 
algorithms,  providing  upgrades  in  the  stand-alone  version  of  the  algorithm,  and 
presenting  the  results.  A  technical  discussion  of  the  progress  with  the  Day  and  Night 
cloud  algorithms  is  given  in  Sections  6  and  7  respectively.  A  discussion  of  the  actual 
deliveries  is  given  in  this  Section  5,  along  with  a  discussion  of  other  minor  areas 
addressed  under  this  category  but  not  discussed  in  Sections  6  and  7. 
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There  were  several  deliveries  related  to  this  task.  At  a  meeting  in  December  2004,  we 
presented  the  results  of  analysis  of  the  Day  cloud  algorithm  done  primarily  under  the 
previous  funding  (Shields  et  al.  2007).  At  this  meeting,  we  were  asked  to  concentrate 
first  on  two  aspects  of  the  night  algorithm,  namely  how  we  could  ground-truth  the  results, 
and  how  we  could  move  to  a  high  resolution  night  algorithm  in  the  future.  In  response, 
we  developed  and  presented  methods  for  ground-truthing  of  the  night  cloud  algorithm 
based  on  the  beam  transmittance  to  bright  stars,  and  presented  this  infonnation  in  the 
April  2005  meeting.  Also,  we  extracted  sample  backgrounds  and  developed  the  concepts 
for  a  full  resolution  night  algorithm,  as  documented  in  Memo  AV05-012t,  and  presented 
at  the  same  meeting.  The  results  will  be  discussed  in  Section  7,  and  the  Power  Point 
presentation  may  be  provided  to  the  sponsors  upon  request.  At  the  sponsors’  request,  we 
also  provided  a  full  review  of  the  concepts  behind  the  day  and  the  night  algorithms  at  a 
June  2005  meeting. 

We  were  asked  to  concentrate  next  on  processing  a  set  of  both  day  and  night  1 -minute 
resolution  data  from  SOR  from  July  and  August  2005,  so  that  it  could  be  used  by  SOR, 
TASC,  and  MPL  for  forecasting  studies.  This  required  updating  the  stand-alone 
algorithm  and  utility  programs  for  compatibility  with  the  SOR  systems,  and  integrating 
the  night  algorithm  into  the  code,  as  documented  in  Technical  Memos  AV05-033t.  The 
update  to  the  night  geometric  calibration  is  documented  in  Memo  AV05-034t  and  AV05- 
03  5t.  The  processing  required  extracting  the  day  clear  sky  libraries  and  other  inputs 
required  to  run  the  data.  We  processed  approximately  15,000  day  images  and  3700  night 
images.  The  processing  is  documented  in  Memo  AV05-032t,  and  the  format  of  the 
results  is  documented  in  Memo  AV05-031t.  This  memo  and  the  data  were  delivered  to 
SOR  and  TASC  on  1 1  October  2005.  This  was  our  first  opportunity  to  evaluate  the  night 
algorithm  results  on  a  substantial  data  set. 

During  this  time,  we  also  worked  on  methods  for  better  ground-truthing  of  both  day  and 
night  cloud  algorithm  results,  as  documented  in  Memo  AV05-021t.  As  indicated  in  the 
SOW,  we  had  originally  intended  to  use  lidar  for  this  purpose.  However,  initially  the 
lidar  data  were  not  available  to  us,  and  then  when  they  were  delivered,  they  were  in  a 
form  we  were  not  yet  in  a  position  to  use.  Eventually,  it  was  decided  by  our  sponsors  that 
the  other  work  discussed  below  was  more  important  at  the  time. 

Instead  of  the  lidar  comparison,  a  program  to  enable  systematic  assessment  of  algorithm 
accuracy,  called  SORCloudAssess,  was  developed  and  applied  to  this  test-bed  data.  The 
results  for  the  night  algorithm  are  documented  in  Memo  AV05-037t,  and  the  results  for 
the  day  algorithm  are  documented  in  Memo  AV05-038t.  Although  this  program  relied  on 
visual  assessment  of  the  results,  it  at  least  provided  a  consistent  and  systematic  way  to 
assess  the  results. 

We  found  the  day  algorithm  to  provide  correct  results,  based  on  visual  assessment  of  the 
imagery,  approximately  97.4%  of  the  time.  (If  sunset  is  not  counted,  the  results  were 
accurate  98.4%  of  the  time,  based  on  the  visual  assessment.)  At  night,  the  results  in  the 
line  of  sight  were  accurate  approximately  87.5%  of  the  time.  This  lower  value  is  partly 
because  the  night  algorithm  was  a  moderate  resolution  algorithm  (based  on  contrast).  An 
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additional  test,  which  evaluated  whether  the  answers  were  correct  in  the  general  region  of 
the  line  of  sight  yielded  an  estimated  accuracy  of  95.0%.  These  results  were  reported  in 
the  December  2005  meeting. 

We  also  began  work  to  develop  forecast  testing  techniques,  as  documented  in  Memo 
AV05-036t.  Programming  on  this  technique  was  begun,  although  we  did  not  finish  the 
work  due  to  other  higher  priorities.  We  reported  on  this  work  at  the  February  2006 
meeting,  and  also  recommended  that  rather  than  predicting  yes/no  whether  the  line  of 
sight  would  be  cloud-free,  we  provide  a  probability,  and  thus  an  ability  to  rank  each  site. 

In  the  December  2005  meeting  we  presented  the  results  of  a  preliminary  analysis  of  night 
data  taken  in  Virginia,  in  a  very  bright  urban  location.  Following  an  initial  analysis 
documented  in  Memo  AV04-054t,  we  found  that  there  was  too  much  scattering  of  city 
light  to  easily  detect  stars  over  much  of  the  image.  As  a  result,  we  adjusted  the 
instrument  to  acquire  spectral  data  at  night.  An  analysis  of  this  data  showed  that  the 
clouds  are  very  easily  detectable  visually  in  the  image,  and  the  typical  cloud  signal  is 
more  than  a  factor  of  10  brighter  than  the  clear  sky  signal.  We  concluded  that  the  raw 
data  show  that  an  algorithm  could  be  developed  for  this  very  bright  location  or  similar 
locations  if  needed. 

The  next  algorithm  question  we  were  asked  to  tackle  was  to  evaluate  whether  the  day 
algorithm  would  work  in  very  hazy  locations,  using  Virginia  data  as  a  test  bed.  For  this 
processing,  we  updated  the  algorithm  to  enable  using  the  Near  Infrared  (NIR)  data,  which 
are  expected  to  do  better  in  hazy  environments.  Data  from  April  2005  were  processed, 
because  it  was  the  closest  we  had  to  summer,  when  the  conditions  should  be  worst.  The 
results  were  presented  in  the  Feb  2006  meeting.  (We  were  not  asked  to  deliver  the  data.) 
The  data  from  1-hour  intervals  were  processed,  which  yielded  a  data  set  of  307  images  - 
perhaps  not  enough  to  be  statistically  significant,  but  enough  to  detennine  whether  the 
algorithm  works  reasonably  in  hazy  environments.  Based  on  the  systematic  visual 
assessment  using  the  SORCloudAssess  program,  we  found  that  if  the  sunset  data  are  not 
included,  the  data  appear  to  be  correct  about  98.1%  of  the  time  in  this  test  set.  The 
processing  and  results  were  documented  in  Memo  AV06-018t. 

During  this  time,  we  also  worked  on  getting  the  algorithms  ready  to  field  with  the 
instruments.  This  work  is  discussed  in  Section  5.4.  This  work  completes  Primary  Task  1 
in  the  SOW. 

5.2.  Analysis  of  Infrared  Sensors 

Task  2  under  the  SOW  was  to  analyze  the  possible  uses  of  IR  systems  for  this  cloud 
application.  Most  of  this  work  was  completed  under  the  previous  contract,  and  is 
reported  in  detail  in  Shields  et  al.  2007  and  in  Memo  AV07-026t.  An  overview  of  the 
results  is  presented  in  Section  8  below.  The  presentation  was  prepared  under  this 
contract,  and  presented  in  December  2004.  This  work  completed  Primary  Task  2  in  the 
SOW. 
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5.3.  Evaluation  of  System  Upgrades 

Task  3  under  the  SOW  was  to  evaluate  possible  system  upgrades.  We  evaluated  several 
upgrades,  including  replacing  the  Accessory  Control  Panel  with  a  much  simpler  (but 
slightly  less  flexible)  system.  We  evaluated  the  current  cameras  and  lenses  that  could  be 
used,  and  their  impact  on  required  cooling.  Some  of  this  work  was  reported  in  the 
December  2004  presentation,  and  is  discussed  in  Section  9.1.  This  work  completed 
Primary  Task  3  in  the  SOW.  Following  the  decision  to  refurbish  the  WSI  which  became 
available,  we  concentrated  more  of  our  time  on  preparing  these  units,  as  discussed  in  the 
later  sections. 

5.4.  Refurbishment  of  WSI  Units  for  Field  Deployment  Support  and  Maintenance 
Support 

Early  in  the  contract,  we  were  asked  to  include,  under  Task  3,  to  begin  refurbishment  of 
existing  WSI  systems  for  support  of  SOR  field  deployments.  The  Optional  funds  were 
also  directed  to  a  large  extent  toward  this  task,  as  well  as  toward  maintenance.  The 
system  maintenance  is  documented  in  Section  9.2.  The  hardware  and  software 
refurbishments  are  documented  in  Sections  9.3  and  9.4  respectively,  and  the  site  logistics 
support  is  documented  in  Section  9.5.  A  brief  overview  is  provided  in  this  section. 

5.4.1.  WSI  Refurbishment 

The  first  two  units  for  refurbishment  were  received  in-house  in  February  2005,  as 
documented  in  Technical  Memo  AV05-004t.  The  retired  unit  that  was  already  in-house 
is  documented  in  Memo  AV05-020t.  Other  units  that  can  serve  as  test  units  and  spare 
parts  are  documented  in  Memos  AV05-025t,  AV05-028t,  and  AV06-022t.  These 
instruments  were  originally  built  for  the  Department  of  Energy  (DOE)  and  used  for  many 
years  at  a  variety  of  sites.  They  were  retired  beginning  in  late  2004.  We  proposed  that 
the  retired  instruments  be  given  to  MPL  for  use  in  other  programs,  and  this  was  done. 

Typical  refurbishment  tasks  included  disassembly,  replacing  worn  components  such  as 
the  coolant  tubing,  replacing  any  failed  components  such  as  arc  drive  motors,  and  getting 
the  cameras  tested  and  purged  at  Photometries.  We  also  replace  filters  as  necessary, 
replace  the  shutter,  replace  cables  as  required,  clean  the  system,  replace  optical  domes, 
and  re-label  connections.  The  occultor  shades  were  replaced  with  shades  appropriate  to 
the  new  locations. 

The  systems  did  not  include  processing  computers  for  real-time  algorithm  processing. 
We  added  processing  computers,  updated  the  GPS  system,  and  also  added  components 
such  as  and  external  300  GB  drive  and  a  power  switch  to  enable  automatic  reboots. 

The  systems  are  refocused  and  radiometrically  calibrated.  The  electronics  such  as  the 
meters  and  the  occultor  arc  drive  control  are  recalibrated.  The  systems  are  thoroughly 
tested  prior  to  deployment.  The  first  system  entered  test  in  June  2005.  We  had  hoped  to 
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deploy  the  first  system  under  this  contract;  however  the  sponsor-provided  site  was  not 
ready  in  time.  Both  the  second  and  the  third  units  were  partially  refurbished  under  this 
contract. 

These  developments  were  reported  in  the  talks  on  an  ongoing  basis,  and  are  further 
discussed  in  Section  9.3. 

5.4.2.  Software  Upgrades  for  the  Refurbished  WSI’s 

The  refurbished  WSI  systems  have  quite  different  control  computer  systems,  because 
they  run  on  DOS.  This  is  because  the  Photometries  Camera  used  in  those  systems,  the 
Series  200,  requires  a  DOS  operating  system.  SOR  evaluated  the  cost  of  upgrading  the 
cameras  to  the  Series  300,  and  upgrading  the  computer  systems,  and  decided  to  stay  with 
the  DOS  system.  The  code  was  somewhat  different  in  terms  of  data  acquisition  -  for 
example,  they  acquired  a  full  data  set  every  6  minutes,  and  red  or  open  hole  filter  every  2 
minutes,  whereas  for  the  SOR  task,  we  needed  every  1  minute  during  the  daytime,  and  2 
minutes  at  night.  These  systems  also  did  not  have  any  real-time  algorithms.  A  very 
extensive  rework  of  the  software  was  accomplished,  both  in  order  to  adapt  the  system  to 
the  SOR  needs,  and  in  order  to  make  the  system  more  robust  and  easier  to  perform 
routine  QC.  These  developments  were  reported  in  the  talks  on  an  ongoing  basis,  and  are 
further  discussed  in  Section  9.4. 

5.4.3.  Logistics,  Testing,  and  Deployment 

As  part  of  the  options,  we  were  asked  to  prepare  the  instruments  for  deployment. 
Preparation  of  the  actual  sites  was  being  done  by  SOR  and  by  another  group  under 
contract  to  SOR.  We  worked  with  these  two  groups  on  the  deployment  and  site  logistics 
regarding  the  required  site  support  and  deployments  during  early  2005,  and  the  logistics 
were  mostly  worked  out  by  May  2005. 

5.4.4.  Field  Repairs 

The  field  repairs  are  discussed  in  Section  9.2.  The  SOR  unit,  Unit  12,  which  was 
originally  delivered  in  January  1999,  required  several  repairs,  but  it  was  kept  running 
most  of  the  time.  The  Virginia  unit,  Unit  14,  which  was  built  in  2000,  was  kept  running 
through  most  of  2005  and  2006,  however  there  were  not  sufficient  funds  to  repair  the 
camera  when  it  failed  in  March  2006. 

We  believe  that  the  work  documented  in  Section  5,  and  further  documented  below,  meets 
and  exceeds  the  requirements  given  in  the  Statement  of  Work. 

6,  Day  Algorithm  Upgrade  and  Analysis 

As  discussed  in  Section  5,  there  was  much  more  emphasis,  from  our  sponsors,  on 
evaluating  and  checking  the  day  algorithm  than  on  developing  it  during  this  contract. 
While  Section  5  discussed  the  deliveries  and  time-lines,  this  section  will  provide  more 
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detail  on  the  results.  The  integration  of  the  algorithms  into  the  field  software  is  discussed 
in  Section  9.4. 

6.1.  Previous  Day  Algorithm  Results 

Our  first  in-depth  analysis  of  the  day  algorithm  was  done  under  funding  from  another 
sponsor,  and  was  reported  to  SOR  in  December  2004.  This  analysis  was  done  with  data 
from  a  site  in  Oklahoma,  and  the  results  are  reported  in  Shields  et  al.  2007.  In  general  we 
were  quite  pleased  with  the  results.  The  algorithm  handled  opaque  clouds  and  thin 
clouds  quite  well.  We  did  find  that  there  was  enough  variation  in  haze  and  its  impact  on 
the  algorithm,  that  it  would  be  helpful  to  develop  an  adaptive  algorithm  to  handle  haze. 

An  adaptive  algorithm  had  already  been  developed  for  the  Daylight  Visible/NIR  WSI 
(Day  WSI),  described  in  Shields  et  al.  2003d.  This  algorithm  for  the  Day  WSI  uses  the 
NIR/blue  ratio,  which  is  less  influenced  by  aerosols  in  very  hazy  environments,  and  also 
makes  adjustments  for  the  aerosol  amount.  At  the  meeting  in  June  2005,  we  gave  a  fairly 
detailed  overview  of  the  Day/Night  WSI  (D/N  WSI)  algorithm  used  during  the  daytime, 
including  these  updates  to  the  Day  WSI  algorithm. 

6.2.  Algorithm  Results  for  Data  from  the  SOR  Site 

To  further  evaluate  the  day  algorithm  used  with  the  D/N  WSI  (which  was  based  on  the 
red/blue  ratio,  and  did  not  include  the  adaptive  feature),  we  processed  a  test-bed  of  data 
from  July  and  August  2005  taken  at  1 -minute  intervals  at  the  SOR  site.  It  should  be 
noted  that  at  this  time  we  programmed  a  version  of  the  adaptive  algorithm,  but  did  not 
find  the  results  consistent  enough  to  use  routinely.  It  has  not  yet  been  determined 
whether  this  is  due  to  software  bugs,  less  than  optimal  input  parameters,  or  a  need  for  a 
more  sophisticated  approach  than  we  used. 

A  set  of  15,000  images  were  processed  and  analyzed.  The  associated  documentation 
memos  are  listed  in  Section  5.1  and  in  the  Reference  section.  Sample  results  are  shown 
in  Figures  1-4.  In  each  of  these  figures,  the  raw  image  on  the  left  is  the  red  image.  The 
cloud  decision  image  on  the  right  shows  the  results  of  the  cloud  algorithm.  In  this  image, 
black  is  the  color  code  used  for  “no  data”,  blue  is  “no  cloud”,  yellow  is  pixels  the 
algorithm  has  identified  as  “thin  cloud”,  and  white  is  “opaque  cloud”.  The  texture  within 
each  of  these  clouds  is  only  to  help  the  analyst  assess  the  images.  (For  example,  in  the 
opaque  cloud  regions,  colors  ranging  from  grey  to  white  indicate  how  much  the  ratio 
exceeded  the  opaque  threshold.  Similarly,  the  coloration  within  the  blue  and  the  yellow 
regions  have  to  do  with  variations  within  the  clear  and  thin  cloud  determination 
schemes.) 

Because  the  WSI  looks  up,  the  directions  on  these  images  are  not  the  same  as  on  a  map. 
East  is  to  the  right,  but  North  is  at  the  bottom  of  the  image.  (Visualize  the  scene  lying  on 
your  back  with  your  toes  to  the  north.) 
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Fig.  1.  Raw  red  and  processed  cloud  decision  from  SOR  site,  26  Jul  05  2200 


Fig.  2.  Raw  red  and  processed  cloud  decision  from  SOR  site  9  Jul  05  1700 

We  felt  that  the  day  algorithm  did  quite  well  for  this  SOR  data  set  illustrated  in  Figures  1 
through  4.  Figure  1  was  chosen  because  it  is  fairly  typical,  and  it  also  illustrates  the 
ability  of  the  algorithm  to  identify  opaque  clouds  properly  in  the  solar  aureole  region  and 
near  the  horizon.  Note  particularly  the  small  clouds  that  have  been  correctly  identified 
near  the  horizons  to  the  north  and  south,  as  well  as  the  thin  clouds  in  the  south-east  of  the 
image. 

Figure  2  was  selected  to  demonstrate  the  ability  to  identify  small  clouds,  as  well  as 
optically  thin  clouds  (shown  in  yellow).  Figure  3  was  selected  to  demonstrate  the  ability 
to  identify  both  dark  and  bright  clouds  (brightness  can  be  seen  in  the  raw  image;  both 
were  shown  in  white  in  the  cloud  decision  image).  Note  particularly  the  dark  cloud  to  the 
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south  of  the  sun,  in  contrast  with  the  bright  clouds  to  the  SW  and  NE  of  the  sun.  Figure  4 
was  selected  to  demonstrate  the  ability  to  identify  high  thin  clouds.  In  Figure  4  there  is  a 
lower  layer  of  opaque  clouds  (shown  in  white),  and  a  high  very  thin  layer  of  clouds 
(shown  in  yellow),  particularly  to  the  west  of  the  zenith,  that  was  properly  identified. 
(The  presence  of  these  thin  clouds  in  the  raw  images  is  easier  to  see  when  the  image  are 
viewed  in  an  image  processing  program  than  in  a  word  document  such  as  this  report.) 


Fig.  3.  Raw  red  and  processed  cloud  decision  from  SOR  site  26  Jul  05  2100 


Fig.  4.  Raw  red  and  processed  cloud  decision  from  SOR  site  3 1  Jul  05  1500 

6.3.  Systematic  Evaluation  of  the  SOR  Results 

Although  general  impressions  are  helpful,  we  wanted  to  develop  a  method  to 
systematically  analyze  the  algorithm  results.  We  have  had  many  opportunities  over  the 
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years  to  view  the  real  sky,  and  immediately  after,  view  what  the  acquired  image  looks 
like.  As  a  result  of  this  experience,  we  feel  that  we  are  able  to  make  a  reasonable 
assessment  of  whether  clouds  are  present  from  the  raw  images.  Given  these  raw  images, 
we  can  make  a  visual  assessment  of  whether  the  algorithm  is  correct  by  comparing  the 
raw  and  cloud  decision  images.  In  difficult  cases,  we  can  look  at  motion  within  the 
images,  compare  the  different  spectral  filters,  and  also  view  the  raw  data  at  different 
contrast  and  brightness  levels.  Although  this  visual  determination  is  not  perfect,  we  felt  it 
was  a  good  first  step  in  assessing  how  often  the  algorithm  had  problems,  and  in  which 
situations.  The  program  SORCloudAssess  was  written  to  allow  us  to  do  this  in  a 
systematic  way. 

The  display  for  the  program  SORCloudAssess  is  shown  in  Figure  5.  In  this  program, 
several  Regions  of  Interest  (ROI)  are  marked,  at  the  zenith,  azimuth  angle  combinations 
(0,0),  (30,0),  (30,90),  (30,180),  (30,270),  (60,0),  (60,90),  (60,180),  (60,270),  (80,90),  and 
(80,270).  The  analyst  uses  the  mouse  to  mark  any  of  these  predesignated  ROI’s  that  are 
judged  to  be  incorrect,  and  also  indicates  whether  the  overall  results  are  judged  to  be 
correct  over  90%  of  the  image.  (The  ROI  indicators  are  difficult  to  see  in  the  report 
format,  and  are  designed  for  best  viewing  during  use  of  the  program.) 


Fig.  5.  SORCloudAssess  program  display  for  user  assessment  of  algorithm  results 

As  noted  above,  there  are  several  caveats.  We  realize  that  this  program  uses  a  visual 
assessment.  We  feel  that  the  visual  assessment  is  quite  good  because,  if  necessary,  we 
can  leave  the  program  to  view  the  images  with  different  contrast  enhancements,  view  the 
other  filters,  and  view  movies  to  assess  cloud  motion.  Still,  this  is  not  an  absolute  ground 
truth.  It  is  intended  to  allow  those  developing  the  algorithm  to  determine  where  the 
current  problems  are,  and  to  look  at  whether  algorithms  are  improving. 
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We  assessed  data  taken  at  hourly  intervals  from  the  entire  test  bed  data  set,  thus  assessing 
310  images,  or  3410  ROI’s.  Overall,  we  found  that  the  ROI  results  were  evaluated  to  be 
correct  97.4%  of  the  time,  with  a  99.0%  rate  at  the  zenith,  and  a  95  -  96%  rate  near  the 
horizon.  Later  evaluation  sorted  out  the  sunrise/sunset  cases,  and  with  these  removed,  the 
average  results  over  the  full  sky  were  evaluated  to  be  correct  98.4%  of  the  time.  Results 
were  estimated  to  be  correct  over  90%  of  the  sky  94.5%  of  the  time.  The  distribution  of 
the  results  as  a  function  of  the  ROI  position  is  shown  in  Figure  6. 


Fig.  6.  Fraction  of  correct  answers,  in  percent,  for  each  ROI  for  SOR  Day  Test  Bed  Processing 

We  also  evaluated  when  the  errors  occurred.  Of  the  310  hourly  images  examined,  293 
(94.5%)  passed  the  test  regarding  whether  it  was  evaluated  to  be  correct  over  90%  of  the 
image.  Ten  cases  (3.2%)  failed  because  the  algorithm  is  not  yet  optimized  for  dawn  or 
dusk.  Six  cases  (1.9%)  failed  in  identifying  thin  cloud.  One  case  (0.3%)  had  a  problem 
in  the  raw  data.  Of  the  23  days  evaluated,  6  mornings  and  2  evenings  had  problems.  The 
other  sunrise  and  sunset  cases  were  Ok. 

Examples  of  the  few  failures  are  shown  in  Figures  7-9.  Figure  7  shows  an  example 
where  the  thin  cloud  identification  was  poor,  and  not  all  of  the  thin  clouds  were 
identified.  (These  may  be  seen  as  the  light  blue  regions  that  are  shaped  like  cloud  but  not 
colored  yellow.)  Figure  8  shows  the  same  example  when  run  with  the  adaptive 
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algorithm.  In  this  case,  the  results  of  the  adaptive  algorithm  were  excellent  (the  yellow 
regions  correspond  well  with  the  thin  clouds  apparent  in  the  raw  image).  Other  cases  run 
with  the  adaptive  algorithm  were  not  good,  and  we  have  not  yet  fully  debugged  this 
feature,  due  to  other  sponsor  priorities.  Figure  9  shows  a  poor  result  at  sunrise.  Here  the 
enhanced  redness  of  the  sky  associated  with  the  long  path  length  results  in  a  higher 
red/blue  ratio,  and  results  in  the  incorrect  identification  of  portions  of  the  clear  sky  as 
cloud. 


Fig.  7.  Example  showing  failure  to  detect  thin  clouds 


Fig.  8.  Same  image  as  Fig.  7,  but  run  with  the  adaptive  algorithm  turned  on 
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Fig.  9.  Example  of  failure  of  algorithm  at  sunrise 


Fig.  10.  Typical  results  showing  ability  to  detect  clouds  near  the  horizon 

At  a  previous  meeting,  an  analyst  from  another  group  had  stated  the  opinion  that  the  WSI 
algorithm  would  never  be  able  to  handle  either  cloud  motion  or  clouds  near  the  horizon. 
As  a  result,  we  looked  at  these  two  categories  specifically.  We  found  only  one  ROI  out 
of  the  3410  cases  that  was  affected  by  cloud  motion.  Except  in  the  case  of  very  fast 
moving  clouds  with  very  large  changes  in  radiance,  the  MPL  algorithm  should  not  have  a 
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problem.  The  algorithm  was  originally  designed  (in  the  early  1980’s)  so  that  the  location 
of  the  clouds  in  the  cloud  decision  image  will  be  the  same  as  the  location  of  the  clouds  in 
the  red  image.  If  cloud  motion  becomes  a  problem,  we  are  aware  of  algorithm  upgrades 
that  could  be  made  to  address  this. 

Regarding  the  near-horizon,  as  noted  earlier,  we  found  that  the  results  were  not  quite  as 
good  at  the  horizon  (95-96%  estimated  accuracy),  but  in  general  they  were  quite  good.  A 
typical  sample  is  shown  in  Figure  10.  This  reasonable  result  at  the  horizon  is  partly  due 
to  an  “indeterminate  cloud”  category  that  will  be  discussed  later.  The  indeterminate 
cloud  logic  was  added  to  the  algorithm  in  the  middle  1980’s  and  has  worked  well  in  a 
variety  of  environments. 

As  noted  earlier,  we  programmed  a  version  of  the  adaptive  algorithm  for  the  Day/Night 
WSI.  In  isolated  cases,  such  as  Figures  7  and  8,  it  provided  significantly  improved 
results.  However,  we  were  not  satisfied  with  the  results  in  general,  and  did  not  have  the 
time  at  that  point  to  detennine  the  source  of  the  problems.  As  a  result,  the  analysis 
discussed  here  was  all  done  with  the  adaptive  algorithm  turned  off. 

The  work  with  the  SOR  test  bed  data  was  reported  in  the  December  2005  talk,  and  further 
examples  are  provided  there. 

6.4.  Evaluation  of  Data  from  the  Hazier  Virginia  Site 

Because  the  SOR  site  is  a  clear  desert  air  enviromnent,  we  expect  it  to  be  a  relatively  easy 
environment  for  the  algorithm.  We  therefore  decided  next  to  process  a  test-bed  of  data 
from  the  Virginia  site,  which  can  be  extremely  hazy,  especially  in  the  summer.  For  this 
analysis,  we  processed  data  from  April  2005,  as  it  was  the  closest  month  we  had  to 
summer,  when  the  problem  should  be  worst.  Hourly  data  were  processed,  and  307 
images  were  analyzed.  The  results  were  documented  in  Technical  Memo  AV06-018t  and 
reported  in  the  February  2006  meeting,  and  more  details  are  presented  there.  For  this 
Virginia  data  set,  using  the  SORCloudAssess  program,  we  estimate  the  overall  algorithm 
accuracy,  if  sunrise/sunset  is  excluded,  at  approximately  98.1%,  with  one  caveat 
discussed  below.  The  full  image  was  estimated  to  be  over  90%  correct  in  96.7%  of  the 
cases. 

For  this  site,  we  went  ahead  and  programmed  the  NIR/blue  algorithm.  The  NIR  filters 
were  installed  in  the  WSI  systems  beginning  in  1997,  because  we  felt  that  they  should 
enhance  the  contrast  between  the  aerosols  (i.e.  small  droplets  with  size  near  0.1  pm)  and 
the  thin  clouds  (large  droplets  with  size  near  1-10  pm.  The  aerosol  scattering  decreases 
quickly  as  a  function  of  wavelength  near  the  red  and  NIR  filter  wavelengths  of  650  and 
800  pm,  with  the  rate  of  decrease  depending  on  droplet  size  distribution  and 
characterized  by  the  Angstrom  coefficient.  The  thin  clouds,  having  larger  droplets, 
scatter  effectively  at  the  longer  wavelengths  as  well  as  the  shorter  wavelengths.  The 
theory  is  shown  in  more  detail  in  the  July  2004  talk. 


17 


We  had  not  yet  optimized  the  adaptive  algorithm,  because  we  were  asked  to  delay  the 
algorithm  upgrade  in  order  to  assess  the  Virginia  data.  During  the  processing,  we  found 
that  there  were  4.5  days  with  very  hazy  results,  and  we  simulated  the  adaptive  algorithm 
by  changing  the  adaptive  parameter  for  these  4.5  days  only. 

Several  examples  of  the  results  for  the  Virginia  test  bed  data  are  shown  in  Figures  1 1 
through  15.  These  images  are  shown  in  the  same  format  as  those  from  SOR,  however  the 
raw  imagery  in  this  case  are  the  NIR  images. 


Fig.  11.  Raw  NIR  and  cloud  decision  from  Virginia  site  15  April  05  1900 


Fig.  12.  Raw  NIR  and  cloud  decision  from  Virginia  site  12  April  05  1500 

Figure  1 1  is  a  fairly  typical  example.  It  identifies  both  the  thin  and  the  opaque  clouds  of 
varying  size  and  location  in  the  sky  quite  well.  Figure  12  was  chosen  to  demonstrate  the 
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detection  of  cirrus  clouds  and  contrails.  Again,  the  results  are  quite  good.  However,  in 
this  image,  there  are  some  yellow  speckles  in  the  blue  sky  regions.  This  is  due  to  the 
spectral  dependency  of  the  fiber  optic  taper  used  in  the  camera,  and  should  be  easy  to 
remove  with  a  calibration  correction.  This  calibration  correction  is  one  of  the  features  we 
hope  to  add  to  the  algorithm  in  the  future.  Figure  13  illustrates  a  result  with  broken 
clouds,  and  we  feel  this  result  is  quite  good. 


Fig.  13.  Raw  NIR  and  cloud  decision  from  Virginia  site  02  April  05  2200 


Fig.  14.  Raw  NIR  and  cloud  decision  from  Virginia  site  18  April  05  1600 

Figures  14  and  15  were  chosen  from  the  4  Vi  days  with  more  haze.  In  these  images,  the 
horizon  and  the  solar  aureole  have  been  identified  by  the  algorithm  as  “indeterminate”. 
These  regions  are  colored  grey  in  the  cloud  decision  image.  The  indeterminate  regions 
are  where  the  algorithm  cannot  distinguish  between  clear  sky  and  cloud  due  to  the  haze. 
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The  algorithm  automatically  detects  that  the  distinction  between  haze  and  cloud  for  this 
region  is  not  large  enough,  and  identifies  the  pixels  accordingly.  Visually,  this  is  similar 
to  a  case  where  the  haze  is  so  heavy  and  so  white  that  one  cannot  see  the  clouds,  except 
that  by  using  the  NIR  filter,  the  WSI  can  “see”  clouds  that  may  not  be  seen  in  the  haze  by 
the  visual  observer. 


Fig.  15.  Raw  NIR  and  cloud  decision  from  Virginia  site  19  April  05  1500 

We  should  add  that  the  haze  does  attenuate  the  transmission  in  the  visible.  However, 
because  we  are  more  interested  in  transmission  at  wavelengths  in  the  Short  Wave  IR  near 
1 .6  pm,  we  normally  do  not  want  to  identify  this  haze  as  a  cloud.  However,  it  may  be 
that  the  haze  could  somewhat  affect  the  transmission  in  the  SWIR.  As  a  result,  in  the 
future,  we  plan  to  add  features  to  the  algorithm  to  enable  it  to  identify  heavy  haze  cases, 
and  potentially  use  this  information  in  assessing  the  relative  ranking  of  different  sites. 

The  distribution  of  results  as  a  function  of  ROI  is  shown  in  Figure  16.  Although  there  are 
obvious  improvements  we  can  make  with  the  algorithm,  we  were  very  pleased  with  these 
results.  A  summary  of  these  daytime  results  for  both  the  SOR  and  the  VA  test  bed  data 
sets  is  shown  in  Table  1. 


Table  1 

Summary  of  Estimated  Accuracy  for  the  Day  Cloud  Algorithm 
Using  SORCloudAssess 
For  the  SOR  and  Virginia  Test  Bed  Data  Sets 


Region 

SOR  Results 

VA  Results 

Overall 

98.4% 

98.1% 

Zenith 

99.5% 

97.5% 

Horizons 

98.5% 

98.8% 

90%  Correct? 

96.7% 

96.7% 
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Fig.  16.  Fraction  of  correct  answers,  in  percent,  for  each  ROI  for  Day  Virginia  Test  Bed  Processing 

6.5.  Results  of  a  Comparison  Test  with  a  Ceilometer 

During  this  time,  under  funding  from  another  sponsor,  we  had  the  opportunity  to  run  a 
WSI  beside  a  Vaisala  CT-25k  ceilometer,  designed  to  measure  clouds  to  approximately 
25k  feet,  as  documented  in  Memo  AV05-010t.  This  comparison  was  also  valuable  for 
the  SOR  program  in  two  respects.  First,  it  let  us  evaluate  whether  the  WSI  can  detect  all 
the  clouds  that  a  ceilometer  can,  and  second  it  let  us  evaluate  whether  a  ceilometer 
system  provides  sufficiently  accurate  cloud  fraction  estimates  to  be  useful  for  this 
program.  The  ceilometer  has  the  advantage  that  is  an  active  system,  i.e.  it  puts  out  a  light 
beam,  and  the  backscattering  of  that  light  is  sensed.  This  could  potentially  make  it  more 
sensitive.  The  ceilometer  has  the  disadvantage  that  it  only  senses  in  the  one  direction. 
As  a  result,  the  ceilometer  estimates  cloud  fraction  from  a  temporal  average  over  the  last 
30  minutes,  weighted  more  heavily  for  the  last  10  minutes. 

In  general,  we  found  that  the  ceilometer  and  WSI  agreed  well  for  cases  where  the 
imagery  showed  it  to  be  either  overcast  or  clear.  However,  we  found  that  the  cloud 
fraction  estimated  by  the  ceilometer  was  very  poor  in  other  conditions.  Examples  are 
shown  in  Figure  17.  In  Figure  17,  in  example  a,  the  ceilometer  estimated  sky  cover  to  be 
0  octals,  or  clear,  even  though  the  actual  cloud  cover  is  significant.  In  cases  b  and  c,  the 
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ceilometer  estimated  sky  cover  to  be  1  octal  (1/8  cover),  even  though  the  clouds  cover 
well  over  half  the  sky.  In  case  d,  the  ceilometer  estimated  sky  cover  to  be  3  octals,  and  it 
is  overcast.  In  general,  we  found  that  if  the  sky  is  clear,  the  ceilometer  correctly  reports 
that  result,  but  if  the  sky  cover  was  scattered  or  broken,  it  was  very  often  significantly 
low.  There  were  many  cases  similar  to  those  in  Figure  17  where  the  clouds  either  stayed 
in  one  quadrant  and  were  not  seen  by  the  ceilometer  (as  in  case  a)  or  where  the  clouds 
changed  too  quickly  for  a  temporal  average  to  be  effective.  As  a  result,  we  feel  that  a 
ceilometer  would  not  be  adequate  for  forecasting  that  a  given  line  of  sight  may  soon  be 
blocked,  since  it  does  not  see  the  clouds  outside  its  own  line  of  sight. 


a)  Ceilometer  0/8  b)  Ceilometer  1/8  c)  Ceilometer  1/8  d)  Ceilometer  3/8 


Fig.  17.  WSI  Images  with  simultaneous  ceilometer  report,  examples  where  ceilometer  results  are  not  valid 

Regarding  the  question  of  whether  the  WSI  is  able  to  detect  all  the  clouds  the  ceilometer 
detects,  we  did  a  detailed  examination  of  the  data  taken  every  10  minutes  on  4  of  the  days 
in  the  data  set,  and  we  found  only  one  case  in  which  the  WSI  did  not  appear  to  detect  a 
cloud  detected  by  the  ceilometer.  On  closer  examination,  the  cloud  could  be  seen  in  the 
WSI  image  if  it  was  enhanced  more.  We  did  not  have  sufficient  funding  to  set  up  the 
cloud  algorithm,  so  it’s  possible  that  in  this  one  case,  the  algorithm  might  have  missed 
the  cloud,  even  though  it  can  be  seen  in  the  raw  imagery.  We  did  find  several  cases 
where  the  ceilometer  did  not  detect  the  clouds  even  within  its  line  of  sight,  due  to  the 
height  limitation  of  the  lidar  in  the  ceilometer.  That  is,  the  WSI  was  often  able  to  detect 
clouds  the  ceilometer  could  not  detect. 

6.6.  Summary  of  Day  Cloud  Algorithm  Results  and  Future  Plans 

As  a  result  of  the  day  algorithm  studies  conducted  under  this  contract,  we  concluded  that 
the  WSI  algorithm  is  generally  working  quite  well  in  both  relatively  dry  environments 
and  relatively  hazy  environments.  However,  we  would  like  to  add  adaptive  algorithm 
features  to  identify  the  enhanced  haze  cases  and  adjust  for  them.  There  are  additional 
calibration  corrections  that  can  be  added.  Also,  we  plan  to  do  more  ground-truthing  as 
additional  information  becomes  available,  and  also  work  on  calibrating  the  algorithms  for 
optical  depth. 

In  addition,  at  the  present  time  setting  up  the  algorithm  for  a  given  site  is  very  time 
intensive.  We  made  quite  a  bit  of  progress  during  this  interval  in  making  these  programs 
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easier  to  set  up,  but  much  is  still  needed  in  this  direction.  We  need  to  address  improving 
algorithms  in  the  sunset/sunrise  regime.  And  perhaps  most  importantly,  as  systems  are 
deployed,  we  need  to  set  up  the  algorithms  for  each  of  the  sites,  and  evaluate  how  well 
they  are  doing. 

7.  Night  Algorithm  Developments  and  Analysis 

Under  the  previous  contract,  a  first-version  night  algorithm  had  been  developed,  and  was 
installed  at  SOR  running  in  near-real-time.  This  algorithm  uses  a  very  accurate 
geometric  (angular)  calibration,  and  detects  the  presence  of  approximately  2000  stars  in  a 
clear  sky  image.  It  uses  a  Gaussian  best  fit  for  the  point  spread  function  (PSF)  around  the 
star,  and  bases  the  detection  on  the  contrast  between  the  signal  within  the  PSF  and  the 
background  signal.  The  threshold  contrast  depends  on  location  in  the  sky,  as  well  as 
moon  condition,  and  cloud  category  (opaque  or  thin).  The  algorithm  is  a  moderate 
resolution  algorithm,  making  an  assessment  in  each  of  356  regions  or  cells.  These  cells 
cover  an  increment  of  5°  in  zenith  angle.  From  zenith  angles  of  30  to  90°,  the  azimuth 
increment  for  the  cells  is  also  5°;  for  zenith  angles  of  15  to  30°  the  azimuth  increment  is 
15°,  and  for  zenith  angles  of  0  to  15°  the  azimuth  increment  is  90°.  Typical  cloud 
decision  images  were  presented  in  Shields  et  al.  2007. 

In  addition,  at  that  time  we  had  developed  the  ability  to  determine  an  approximate 
transmittance  value  for  the  earth-to-space  transmittance  at  each  star.  A  sample  of  this 
result  is  also  shown  in  Shields  et  al.  2007.  The  methods  for  extracting  the  measured 
irradiance  from  the  data  are  documented  in  Shields  et  al.  2004a,  and  Shields  et  al.  4007, 
and  other  references  listed  in  these  reports.  Similarly,  the  methods  for  detennining  the 
inherent  irradiance  of  each  star  in  the  WSI  passband  are  documented  in  these  references. 
The  transmittances  are  basically  derived  from  a  ratio  of  the  apparent  irradiances  (ground- 
based  measured)  to  the  inherent  irradiances  (outside  the  atmosphere,  in  this  case 
calculated). 

Following  this  work,  we  began  working  toward  a  night  algorithm  based  on  the  earth-to- 
space  beam  transmittance.  For  this  work,  we  began  developing  software  to  apply  the 
radiometric  calibrations  to  the  night  imagery,  and  then  began  evaluating  the  accuracy  of 
the  transmittance  extraction. 

However,  at  the  December  2004  meeting,  we  were  asked  to  put  two  other  items  higher 
priority.  The  first  of  these  was  development  of  ground-truthing  methods,  and  the  second 
was  development  of  concepts  for  a  high  resolution  night  algorithm.  These  results  were 
reported  at  the  April  2005  meeting  and  are  discussed  in  Sections  7.1  and  7.2.  It  should  be 
noted  that  much  of  the  work  reported  in  Sections  7.1  and  7.2  was  done  under  the  Boeing 
contract  mentioned  in  Section  1,  however  the  work  is  included  here,  as  it  was  sponsored 
by  SOR,  and  has  not  been  reported  in  detail  previously.  The  rest  of  the  work  discussed  in 
Section  7  was  done  under  the  DO  #1 1  that  is  the  subject  of  this  report. 
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7.1.  Ground-truthing  using  Bright  Stars 

Regarding  ground-truthing,  our  goal  is  to  have  some  means  to  verify  that  the  cloud 
algorithm  results  are  reasonably  correct.  We  suggested  that  a  few  of  the  brightest  stars 
not  be  used  in  the  algorithm,  but  that  their  transmittance  be  used  to  provide  a  somewhat- 
independent  check  of  the  algorithm  result  in  the  region  near  the  star.  To  do  this,  we  first 
reviewed  the  current  transmittance  calculations,  made  some  improvements  to  the  logic, 
and  also  removed  most  of  the  variable  stars.  Next,  we  looked  at  how  well  the  measured 
star  irradiances  compared  with  the  inherent  star  irradiances  for  individual  stars. 


Figure  18  shows  a  comparison  between  the  measured  star  irradiance  and  the  inherent  star 
irradiance  as  a  function  of  the  zenith  angle  of  the  star  Alnath.  Methods  similar  in  concept 
to  Langley  plots  were  developed  to  extract  the  aerosol  transmittance.  We  found  that  the 
average  aerosol  transmittance  was  .91,  with  a  range  from  .89  to  .93.  That  is,  a  typical 
aerosol  transmittance  on  a  reasonably  clear  night  for  the  SOR  site  in  the  WSI  open-hole 
passband  is  near  .91.  For  this  study,  we  used  a  value  of  .91  for  all  stars.  As  discussed  in 
Section  7.4  we  later  improved  the  value  for  SOR  based  on  evaluation  of  more  stars,  and 
later  work  shows  that  it  is  site-dependent,  as  would  be  expected.  The  data  corrected  for 
aerosol  transmittance  are  shown  in  Figure  19. 


Zenith  Angle  VS.  Measured  Spectral  Irradiance 
Corresponding  To  Alnath  Location 


Zenith  Angle  VS.  Measured  Spectral  Irradiance 
Corresponding  To  Alnath  Location 


Fig.  18.  Alnath  measured  spectral  irradiance  on  11 
nights. 


Fig.  19.  Alnath  spectral  irradiance  corrected  for 
aerosol  extinction. 


In  Figure  19,  the  dashed  line  is  the  calculated  inherent  irradiance  for  the  star. 
Interestingly,  we  found  a  residual  offset  between  the  measured  and  inherent  star 
irradiances.  The  mean  offset  for  this  limited  data  set  was  near  1.37,  and  the  values 
ranged  from  1.06  to  1.7.  We  are  not  yet  certain  of  the  cause  of  this  offset.  A  fixed  offset 
for  all  stars  could  be  caused  by  a  problem  with  the  WSI  calibration.  From  the  average 
offset,  we  believe  there  may  be  an  issue  with  the  calibration,  and  we  hope  to  investigate 
this  in  the  future. 


The  variation  in  the  offset  from  star  to  star  could  be  caused  either  by  errors  in  the  method 
for  extracting  the  measured  irradiances,  or  by  uncertainties  in  the  library  magnitude  or  by 
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the  use  of  color  temperature.  We  believe  it  is  most  likely  to  be  the  latter.  The  standard 
method  of  characterizing  the  spectral  output  of  the  star  by  color  temperature  is  of  course 
approximate.  We  estimate  a  spectrum  based  on  the  color  temperature,  and  use  this  to 
compute  the  star  inherent  irradiance  for  our  spectral  band.  This  has  some  uncertainty, 
and  may  be  the  cause  of  the  star-to-star  variation  in  offset.  We  decided  that  it  will  be 
necessary  to  develop  a  program  to  automatically  extract  the  correction  constant  for  each 
star  we  use  in  the  star  library,  as  will  be  discussed  in  Section  7.3. 

There  is  also  some  variance,  even  on  a  clear  night,  in  the  apparent  irradiance  corrected 
for  aerosol  transmittance  and  for  the  star  offset.  Typical  STD  values  range  from  5%  to 
15%.  We  evaluated  the  data  as  a  function  of  the  background  brightness,  and  found  that 
the  uncertainty  is  independent  of  background  radiance.  This  indicates  that  the  offset  has 
little  to  do  with  the  ability  of  the  code  to  correctly  extract  the  background.  We  also  found 
no  correlation  between  the  offset  of  a  star  and  an  adjacent  star,  indicating  that  the 
variations  are  not  due  to  small  variations  in  the  clear-night  transmittance.  Our  system  is 
very  well  focused,  with  a  PSF  of  less  than  a  pixel.  This  may  mean  that  under-sampling 
of  the  Gaussian  is  the  cause  of  the  uncertainty.  We  have  not  yet  isolated  the  cause  of  this 
variance.  For  the  bright  star  ground-truthing,  we  selected  those  bright  stars  with  a 
Standard  Deviation  (STD)  in  the  aerosol-corrected  irradiance  of  5%  or  less. 

To  choose  the  ground-truth  stars  for  this  test,  we  started  with  the  brightest  stars,  and 
selected  stars  that  had  multiple  cloud-free  appearances  near  the  zenith,  and  that  also  had  a 
STD  of  5%  or  less  from  the  clear  night  set.  We  next  evaluated  a  number  of  images  to  see 
if  the  corrected  bright  star  transmittances  appeared  to  be  reasonable.  Examples  are 
shown  in  Figures  20  -  25.  These  plots  have  been  corrected  for  the  offsets;  however  they 
have  not  been  adjusted  for  aerosol  transmittance.  That  is,  the  transmittances  include 
losses  due  to  both  clouds  and  aerosols  (and  molecular  losses). 


Fig.  20.  Clear  night,  14  Feb  99,  0340 


Fig.  21.  Cloudy  night,  16  June  99  0510 
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Figure  20  shows  a  clear  night.  The  extracted  transmittances  averaged  .89,  which  is 
reasonable  for  a  clear  night  in  this  wave  band  at  the  SOR  site.  The  average  variation 
from  this  average  was  .04,  and  the  maximum  variation  from  the  average  was  .07. 


Fig.  22.  Broken  clouds,  16  June  99,  0710  Fig.  23.  Broken  clouds,  16  June  99  0730 


Fig.  24.  Broken  clouds,  16  June  99,  0840  Fig.  25.  Broken  clouds,  16  June  99  0850 

A  very  cloudy  case  is  shown  in  Figure  21.  Flere,  the  areas  that  appear  to  be  thinner  are 
identified  with  transmittances  of  .10  and  .14.  Areas  with  thicker  cloud  had  3  stars  that 
were  not  detected,  and  one  star  that  was  detected  with  a  beam  transmittance  of  .01. 
Figures  22  -  25  show  a  sequence  of  images  with  a  fairly  stable,  but  thinning,  broken 
cloud  field.  When  this  sequence  is  viewed  on  the  computer  monitor,  we  can  see  that  the 
cloud  field  in  the  upper  right  quadrant  of  the  image  become  thinner  throughout  this 
period,  because  we  can  see  more  and  more  stars  through  the  clouds  in  the  image.  The 
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transmittance  in  this  cloud  region  slowly  changes  from  about  .16  in  Fig.  22,  to  .26  in  Fig. 
23,  to  about  .40  in  Fig.  24,  and  about  .55  in  Fig.  25.  This  time  series  gives  a  good  sanity 
check  that  the  transmittance  results  are  reasonable.  Also,  in  Fig.  25,  we  can  see  that  the 
regions  that  appear  to  have  thinner  clouds  also  have  higher  transmittances. 

From  this  study,  we  believe  that  the  bright  stars  provide  reasonably  accurate 
transmittances,  with  an  uncertainty  of  about  5%,  over  a  wide  range  of  transmittances. 
Our  plan  is  that  we  will  not  use  these  bright  stars  in  the  algorithm,  but  will  use  them  for 
real-time  checking  of  the  algorithm  results  at  a  future  time. 

7.2.  Concepts  for  a  High  Resolution  Night  Cloud  Algorithm 

The  concepts  for  a  spatially  high-resolution  algorithm  were  first  developed  in  2001 
(although  we  had  informally  been  thinking  about  these  concepts  for  many  years  prior  to 
that  date),  and  are  documented  in  Memo  AV01-069t.  Basically,  the  concept  as  developed 
at  that  time  is  as  follows: 

a)  Extract  typical  radiance  distributions  for  clear  skies  and  for  cloudy  skies 

b)  Use  the  star  detection  and  associated  beam  transmittance  detennination  to  assign  a 
value  of  opaque  cloud,  thin  cloud,  or  no  cloud,  to  selected  star  locations  within  the  image. 

c)  Near  these  star  locations,  determine  how  the  radiance  distribution  differs  from  the 
typical  radiance  distribution  at  the  same  location.  Locations  that  are  determined  to  be 
clear  sky  will  be  used  to  adjust  the  clear  sky  distribution,  and  locations  that  are 
determined  to  be  cloud  will  be  used  to  adjust  the  cloud  sky  distribution. 

d)  Compare  the  radiance  in  each  pixel  with  the  adjusted  nominal  clear  sky  and  cloudy 
sky  distributions  to  detennine  the  presence  of  cloud. 

To  examine  this  concept  further,  we  first  extracted  the  clear  sky  background  from  an 
average  of  145  images  over  1 1  nights.  We  found  that  at  a  given  pixel,  the  variation  from 
night  to  night  on  clear  moonless  nights  was  about  5%  STD.  The  clear  sky  background 
plot  for  the  central  row  and  column  are  shown  in  Figures  26  and  27.  In  these  plots,  the 
black  curve  is  the  average,  and  the  colored  curves  show  specific  nights. 

The  average  image  is  shown  in  Figure  28.  Figure  29  shows  the  signal  as  a  function  of 
hour  angle.  It  is  interesting  that  even  on  a  moonless  night  there  would  be  some 
systematic  variance  with  hour  angle.  It  may  have  to  do  with  how  many  people  keep  their 
lights  on  as  the  night  progresses.  To  the  extent  that  this  variation  is  systematic,  it  is 
predictable  and  therefore  useful  in  the  algorithm  development. 

Figures  26  -  29  show  that  the  clear  sky  background  is  reasonably  well  behaved,  at  least 
in  this  data  set.  We  do  expect  some  variation  with  haze  amount,  which  is  why  it  may  be 
important  to  normalize  the  background  for  a  given  image  as  indicated  in  step  c  above. 
However,  first  we  need  to  know  whether  the  cloudy  radiance  distribution  differs 
sufficiently  from  the  clear  sky  radiance  distribution  for  this  method  to  be  worth  pursuing. 
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SOR  Hour  Angle  Background  Images 
Cross-Section:  Middle  Row 


Fig.  28.  Clear  Sky  background  image  Fig.  29.  Central  Row  plotted  as  a  function  of  hour  angle 

We  extracted  the  cloudy  radiance  distribution  from  101  images  on  5  nights.  Figure  30 
shows  the  data  from  Figure  29,  with  the  cloudy  background  superimposed.  In  this  plot, 
the  color  curves  are  the  clear  sky  background,  and  the  black  curve  is  the  cloudy 
background.  Over  most  of  the  sky,  there  is  a  separation  between  the  clear  and  cloudy 
radiance  levels  of  more  than  100%  of  the  clear  sky  radiance.  Figure  31  shows  those 
regions  with  more  than  100%  separation  colored  in  grey.  The  regions  in  which  the 
separation  between  the  cloudy  and  clear  backgrounds  is  less  than  100%  are  the  relatively 
small  regions  shown  in  black.  Even  in  those  regions,  the  separation  was  significant,  and 
should  be  adequate  for  algorithm  development. 
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Fig.  30.  Cloud  Background  (black  curve)  compared  Fig.  31.  Grey  coloring  indicates  regions 
with  clear  background  (colored  curves)  where  cloud  background  differs  by  over  100% 


Several  examples  of  night  sky  images  and  associated  radiance  levels  are  shown  in  the 
April  05  talk,  and  some  of  these  are  presented  here.  Figure  32  shows  a  clear  night  with 
no  moon.  Figure  33  shows  the  associated  middle  row  radiance  (black  curve)  compared 
with  the  nominal  clear  sky  (blue  curve)  and  opaque  cloud  (red  curve). 


SOR  February  14,  1999  0750  UTC 


SOR  Radiance  Cross  Section  —  Middle  Row 
February  14,  1999  0750  UTC 
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Fig.  32.  Clear  sky  sample,  no  moon 


Fig.  33.  Radiance  from  Fig.  32  compared  with 
nominal  clear  sky  and  opaque  sky  radiances 


Figures  34  and  35  show  similar  examples  with  no  moon,  and  with  broken  cloud.  Note 
that  in  this  example,  the  regions  with  cloud  are  brighter  than  the  clear  sky,  but  darker  than 
the  typical  cloud  curve.  From  the  points  within  the  clouds  that  we  know  are  cloudy 
(based  on  the  transmittance),  we  would  determine  the  cloud  radiance  values  and  lower 
the  nominal  cloud  curve  to  generate  a  cloud  curve  for  that  image,  to  use  in  the  pixel 
evaluation. 
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Fig.  34.  Broken  transparent  cloud  sample 


SOR  Radiance  Cross  Section  —  Middle  Row 
June  16,  1999  0750  UTC 


Fig.  35.  Radiance  from  Fig.  34  compared  with 
nominal  clear  sky  and  opaque  sky  radiances 


From  this  data  and  other  examples,  we  believe  that  the  use  of  the  radiance  distributions 
should  have  a  very  good  chance  of  working  well  under  moonless  conditions.  Clearly, 
there  are  many  details  to  be  developed  and  programmed,  but  we  do  not  see  major  show- 
stoppers  at  this  point. 


To  extend  the  high  resolution  concept  to  moonlight  conditions,  it  will  be  necessary  to 
characterize  the  clear  sky  and  cloudy  backgrounds  as  a  function  of  look  angle,  and  moon 
phase  and  position,  much  as  is  done  with  the  daytime  clear  sky  background  ratios. 
Figures  36  and  37  show  a  clear  night  and  a  cloudy  night,  extracted  when  the  moon  was  at 
similar  positions  and  moon  phases.  In  figure  36,  the  moon  phase  was  .73,  the  relative 
brightness  was  .17,  and  the  zenith  angle  was  59°.  In  Figure  37,  the  moon  phase  was  .64, 
the  relative  brightness  was  .13,  and  the  zenith  angle  was  56°.  The  plots  for  these  two 
nights  are  shown  in  Figure  38,  and  show  excellent  separation.  (The  low  values  in  the 
cloudy  curve  are  from  the  occultor  structure.)  We  expect  that  it  will  be  important  (and 
not  trivial)  to  characterize  the  moon  background  well,  but  that  if  we  can  do  that,  the  high 
resolution  algorithm  should  work  well  under  moonlight  as  well  as  starlight  conditions. 


7.3.  Processing  of  a  SOR  Night  Database 

At  the  April  2005  meeting  where  we  reported  on  the  ground-truthing  and  high  resolution 
concepts,  the  sponsors  were  pleased  with  these  results.  We  were  asked  to  switch 
priorities,  and  work  on  updating  the  night  algorithm  for  the  SOR  site,  and  to  process  and 
evaluate  night  data  for  the  SOR  database  discussed  in  Section  6.  The  algorithm  in  place 
at  the  site  was  the  contrast-based  moderate  resolution  algorithm. 

This  algorithm  requires  that  the  angular  calibration  be  updated  whenever  the  instrument 
is  moved.  The  geometric  calibration  had  not  been  updated  in  quite  some  time,  and  the 
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instrument  had  been  moved  in  the  interim.  As  a  result,  the  night  algorithm  results  that 
were  being  derived  in  the  field  were  very  poor,  as  we  would  anticipate  under  these 
conditions.  (We  would  like  to  automate  the  update  of  the  geo  calibration  in  the  future,  or 
at  least  automate  a  method  to  detect  that  the  instrument  has  been  moved  and  the 
calibration  needs  to  be  updated.) 

SOR  June  23.  1999  0550  UTC  SOR  June  22.  1999  0510  UTC 


Fig.  36.  Nearly  cloud-free  moonlight  case  Fig.  37.  Cloudy  with  similar  moonlight 
image  position  and  moon  phase 


SOR  Radiance  Cross  Section  —  Middle  Row 
Moon  Between  1st  Quarter  and  Full 


Fig.  38.  Middle  row  extracted  from  Figures  36  and  37 

We  extracted  the  current  angular  calibration,  and  updated  the  geometric  algorithm  inputs 
to  the  algorithm,  as  documented  in  AV05-034t  and  -035t.  We  updated  the  stand-alone 
algorithm  to  include  the  night  algorithm,  which  had  previously  been  running  only  in  IDL 
and/or  in  the  field  on  the  Unit  12  system.  We  also  updated  the  horizon  mask,  which 
masks  out  objects  in  the  field  of  view.  There  were  no  other  algorithm  updates  at  this 
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time,  because  we  wanted  to  see  how  well  we  might  expect  the  algorithm  to  perform  when 
the  geometric  calibration  was  up  to  date.  We  processed  the  nighttime  data  from  the  same 
SOR  test-bed  data  from  July  and  August  05  used  for  the  day  algorithm  evaluation.  The 
night  data  were  taken  at  3-minute  intervals  at  that  time,  with  the  result  that  3698  images 
were  processed.  The  results  were  documented  in  Memos  AV05-031t,  AV05-035t,  and 
AV05-037t,  and  delivered  to  the  sponsor  with  the  1-minute  day  data. 

Although  this  work  was  done  with  the  older  algorithm,  because  we  had  not  yet  had  an 
opportunity  to  spend  much  time  on  the  new  algorithm  development,  we  were  pleased  to 
have  an  opportunity  to  process  a  significant  amount  of  data,  and  see  how  this  first  version 
algorithm  behaved.  Several  examples  are  shown  in  Figures  39  through  45.  These  figures 
are  shown  in  the  format  shown  to  the  user  in  the  SORCloudAssess  program.  The  raw 
open-hole  (no  spectral  filter)  image  is  shown  on  the  left,  and  the  cloud  decision  is  on  the 
right.  In  the  cloud  decision  image,  black  indicates  the  “no  data”  category.  Pixels  in  this 
category  can  be  due  to  physical  structures  on  the  horizon,  or  if  the  black  region  is  in  one 
of  the  cloud  decision  cells,  it  means  that  there  were  insufficient  stars  in  the  cell  to  make  a 
determination.  Green  indicates  a  night  thin  cloud  decision,  and  grey-to-white  indicates 
an  opaque  cloud  decision. 


Fig.  39.  Example  of  a  relatively  good  night  clear  sky  result 

In  Figures  39  and  40,  we  see  a  relatively  good  case  and  a  relatively  poor  result  for  a  clear 
sky.  The  most  obvious  problem  with  these  images  is  the  region  impacted  by  bright  lights 
of  Albuquerque,  in  the  lower  left  of  the  image  (regions  shown  incorrectly  in  white  in  the 
cloud  decision  image).  This  region  is  incorrectly  being  identified  by  the  algorithm  as 
opaque  cloud.  Also,  some  other  clear  areas,  particularly  in  the  Milky  Way,  are  identified 
as  thin  cloud  (two  green  cells). 
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Figures  41  and  42  show  two  cases  with  relatively  good  and  relatively  poor  results  for 
overcast.  The  majority  of  the  cells  are  identified  correctly  (opaque  clouds  shown  in 
white -to-grey),  however  there  are  several  cells  that  are  incorrectly  identified  as  thin  cloud 
(green)  or  even  clear  (blue)  in  each  of  these  images. 


Fig.  40.  Example  of  a  relatively  poor  night  clear  sky  result 


Fig.  41.  Example  of  a  relatively  good  night  overcast  sky  result 
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Fig.  42.  Example  of  a  relatively  good  night  overcast  sky  result 


Fig.  43.  Example  of  a  night  broken  cloud  result 

Figure  43  shows  a  broken  cloud  result,  which  had  quite  reasonable  results.  Figure  44 
shows  a  case  with  a  mixture  of  thin  and  opaque  clouds,  and  again  the  results  were 
reasonable  (regions  that  appear  in  the  raw  image  to  be  opaque  are  colored  grey,  and  those 
that  appear  to  be  thin  are  colored  green).  Figure  45  shows  a  case  of  moonlight,  and  here 
the  results  were  also  good.  In  each  of  these  three  cases,  most  of  the  cells  provide 
reasonable  results,  and  a  few  do  not.  Figures  39,  41,  and  43  -  45  were  fairly  typical 
results  for  this  data  set.  (For  Figures  40  and  42  we  tried  to  find  worst  case  results.) 
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Fig.  44.  Example  of  a  night  case  with  opaque  and  thin  clouds 


Fig.  45.  Example  of  a  moonlight  case 

In  order  to  assess  these  results  more  systematically,  the  SORCloudAssess  program  was 
updated  to  allow  assessment  of  night  imagery  as  well  as  day  imagery.  We  made  three 
tests. 

a)  LOS  or  Line  of  Sight  Test.  In  this  test,  we  looked  at  each  of  the  1 1  ROI’s  marked  in 
the  image  on  the  right,  and  marked  it  if  the  result  was  not  correct  for  the  line  of  sight, 
just  as  we  did  with  the  Day  data.  The  results  of  this  test  are  shown  in  Figure  46. 
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b)  ROS  or  Region  of  Sight  Test.  We  recognize  that  with  a  moderate  resolution 
algorithm,  we  would  never  use  exactly  the  Line  of  Sight  to  make  a  decision.  Instead, 
we  would  assess  the  presence  of  clouds  in  the  nearby  region  of  the  line  of  sight. 
Thus,  for  this  test,  we  assessed  the  general  Regions  of  Sight  as  follows.  If  the  line  of 
sight  was  incorrectly  identified  as  cloud,  we  asked  if  there  were  clouds  within  the 
cell,  and  if  there  were,  we  did  not  mark  it  incorrect.  If  the  line  of  sight  was 
incorrectly  identified  as  clear,  we  asked  if  there  was  an  adjacent  cell  identified  as 
cloud,  and  if  there  was,  we  did  not  mark  it  as  incorrect.  That  is,  we  asked  if  the 
answer  was  correct  in  the  region  of  the  line  of  sight.  The  results  of  this  test  are 
shown  in  Figure  47. 

c)  Instead  of  visually  evaluating  whether  the  algorithm  was  correct  over  90%  of  the 
image,  as  was  done  with  the  day  algorithm,  we  assessed  whether  it  was  correct  over 
70%  of  the  image,  because  we  felt  it  would  be  too  difficult  to  assess  a  reduced 
resolution  algorithm  at  the  90%  level. 

The  results  of  Figures  46  and  47  and  a  summary  of  the  night  SORCloudAssess  results  are 
shown  in  Table  2. 


Fig.  46.  Fraction  of  correct  answers,  in  percent,  for  each  ROI  for  SOR  Night  Test  Bed  for  Line  of  Sight 
Test 
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Fig.  47.  Fraction  of  correct  answers,  in  percent,  for  each  ROI  for  SOR  Night  Test  Bed  for  Region  of  Sight 
Test 


Table  2 

Summary  of  Estimated  Accuracy  for  the  Night  Cloud  Algorithm 
Using  SORCloudAssess 
For  the  SOR  Test  Bed  Data  Sets 


Region 

EOS  Results 

ROS  Results 

Overall 

87.7% 

95.5% 

Zenith 

93.1% 

97.9% 

Eastern  Horizon 

87.8% 

99.5% 

Western  Horizon 

70.2% 

81.4% 

70%  Correct? 

95.2% 

In  Figures  46  and  47,  we  can  see  that  the  results  over  most  of  the  sky  are  very  good  for 
the  Region  of  Sight  test  (Fig.  47),  and  reasonably  good  for  the  Fine  of  Sight  test  (Fig. 
46).  This  implies  that  if  we  were  to  continue  to  use  the  reduced  resolution  algorithm  for 
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some  reason,  we  would  have  to  look  at  the  Region  of  Sight,  not  just  the  Line  of  Sight,  to 
assess  the  site. 

The  worst  problem  area  in  each  case  is  the  region  impacted  by  the  bright  lights  of 
Albuquerque.  We  expect  that  this  can  be  significantly  improved  in  the  future, 
particularly  if  we  develop  the  transmittance-based  algorithm.  We  felt  that  for  a  first 
version  of  a  working  night  algorithm,  these  results  shown  in  Figs.  46  and  47  and  Table  2 
were  very  encouraging,  but  they  would  clearly  benefit  from  algorithm  improvements. 

7.4.  Upgrading  the  Night  Algorithm 

Following  delivery  of  the  SOR  night  algorithm  processed  results,  we  immediately  went 
ahead  with  work  toward  integrating  the  beam  transmittance  calculations  into  the 
moderate  resolution  night  algorithm.  Under  this  contract,  we  got  the  algorithm 
programmed  in  IDL,  and  ran  a  few  test  cases. 

As  part  of  this  work,  we  extracted  the  star  inherent  irradiance  offset  for  7600  stars  using 
the  techniques  discussed  earlier.  We  extracted  both  the  correction  factor  and  the  standard 
deviation  in  irradiance,  and  rejected  those  cases  with  standard  deviations  above  15%  (this 
is  an  input  file  variable,  so  it  can  be  changed).  We  found  that  for  this  larger  data  set,  a 
better  aerosol  correction  to  use  was  .87  for  this  SOR  site,  and  used  this  for  all  of  the  stars. 
A  plot  of  the  measured  spectral  irradiance  vs.  the  theoretical  spectral  irradiance  similar  to 
Fig.  20  in  Shields  et  al.  2007  was  regenerated,  with  improved  results.  Work  to 
understand  and  isolate  the  causes  of  the  temporal  variations  in  these  irradiances,  as  well 
as  the  magnitude  and  variations  (from  one  star  to  another)  in  the  correction  factors,  is 
continuing. 

As  reported  in  the  December  2005  talk,  we  used  a  preliminary  IDL  version  of  the 
transmittance-based  algorithm,  and  tested  the  cases  shown  in  Figures  40  and  42,  and 
found  the  results  to  be  better.  An  overview  of  the  first  version  of  the  transmittance-based 
algorithm  is  given  in  Memo  AV06-009t.  Under  the  next  contract,  we  were  asked  to  adapt 
the  transmittance  algorithm  to  one  of  the  instrument  sites,  Site  2,  and  at  that  time  we  went 
ahead  and  converted  it  to  a  high  resolution  algorithm.  This  work  will  be  reported  in 
future  reports. 

7.5.  Evaluation  of  Night  Imagery  at  a  Very  Bright  Site 

Under  the  current  contract,  we  were  also  asked  to  look  at  whether  we  think  it  will  be 
possible  to  use  the  WSI  in  very  bright  environments  at  night.  As  reported  in  Shields  et  al. 
2007,  the  instrument  fielded  in  Virginia  is  in  a  very  bright  environment.  Under  the 
previous  contract,  we  evaluated  the  night  imagery,  and  decided  that  there  was  too  much 
stray  light  to  be  able  to  detect  the  stars  over  enough  of  the  night  sky,  as  documented  in 
Memo  AV04-054t.  As  a  result,  in  June  2004  we  changed  the  setup  to  start  acquiring 
spectral  data  at  night. 
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Under  this  contract,  we  did  further  evaluation  of  the  open  hole  data  taken  prior  to  June, 
and  we  found  that  there  is  a  very  large  contrast  between  data  with  and  without  clouds. 
For  example,  we  compared  two  images  taken  under  starlight,  once  with  a  cloud-free  sky, 
and  one  with  overcast.  The  typical  signal  under  the  cloud-free  conditions  was  near  550, 
and  the  typical  signal  under  the  overcast  was  near  8400,  which  is  about  a  factor  of  15 
brighter.  This  should  be  more  than  enough  signal  difference  to  enable  algorithm 
development.  Examples  of  images  with  clouds  overhead,  and  with  clouds  near  the 
horizon,  are  shown  in  Figures  48  and  49.  In  these  images  we  also  had  very  good  contrast 
between  cloud  and  clear  sky.  Although  we  cannot  detect  many  stars,  the  high  resolution 
algorithm  may  require  many  fewer  stars,  so  the  high  resolution  algorithm  may  be 
adequate,  particularly  with  data  taken  with  the  spectral  filters.  Otherwise,  for  sites  this 
bright,  we  would  have  to  develop  a  new  night  algorithm,  but  there  is  plenty  of  contrast  in 
the  raw  data  with  which  to  develop  such  an  algorithm.  We  do  not  expect  to  use  the 
instrument  at  another  site  this  bright,  so  we  agreed  with  the  sponsors  that  we  will  wait 
and  evaluate  the  light  field  at  the  upcoming  sites  before  making  a  bright-lights  algorithm 
a  priority. 


Fig.  48.  Clouds  overhead  in  a  very  bright  city  Fig  49.  Clouds  near  horizon  in  a  very  bright  city 

We  also  further  evaluated  sunrise  and  sunset.  We  found  that  in  the  SOR  data  set,  out  of 
23  days,  the  results  at  80°  Solar  Zenith  Angle  (SZA)  were  good  on  all  but  6  mornings  and 
2  evenings.  The  test  adaptive  algorithm  improved  results  in  most  cases.  We  also  found 
that  the  opaque  cloud  ratio  appeared  to  be  independent  of  SZA.  This  is  in  contrast  to  the 
results  with  the  Day  WSI,  where  we  did  obtain  improvements  by  making  the  opaque 
threshold  be  SZA-dependent.  We  don’t  know  yet  whether  we  can  develop  the  sunset 
algorithm  so  that  it  always  provides  reliable  results  at  sunset,  although  the  algorithm 
already  provides  reliable  sunset/sunrise  results  the  majority  of  the  time.  This  was  just  a 
preliminary  look.  The  sunset/sunrise  regime  is  an  area  that  will  take  some  effort  if  it 
becomes  a  priority. 
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7.6.  Summary  of  Night  Algorithm  Work 


In  conclusion,  we  made  considerable  progress  under  this  contract  in  developing  and 
assessing  the  capabilities  of  the  night  algorithm,  and  we  have  further  plans  for  significant 
improvements.  Partially  with  funding  from  another  contract,  we  developed  a  concept  for 
ground-truthing  the  cloud  algorithm  results  using  bright  stars,  and  we  further  tested  and 
developed  concepts  for  a  high  resolution  algorithm.  Fully  under  funding  from  the  current 
contract,  DO  #1 1,  we  updated  the  inputs  to  the  SOR  site  contrast-based  night  algorithm, 
and  processed  a  database.  We  tested  these  data  using  Program  SORCloudAssess,  and 
found  that  the  results  were  good,  although  as  we  expected,  they  will  benefit  from  planned 
algorithm  improvements. 

8.  Wavelength  Options  to  use  for  Optical  Cloud  Imaging 

One  of  the  requirements  of  this  contract  was  to  analyze  the  pros  and  cons  of  using 
Infrared  (IR)  systems.  As  discussed  earlier,  the  funding  increment  that  was  sent  at  the 
time  this  priority  was  set  was  used  partially  to  begin  funding  this  contract,  and  was  used 
partially  to  fund  options  in  the  previous  contract.  As  a  result,  we  were  able  to  complete 
this  work  under  the  early  contract.  The  results  were  reported  in  a  talk  in  July  2004,  and 
they  have  been  reported  in  Shields  et  al.  2007  and  in  Memo  AV07-026t.  In  this  report, 
we  will  summarize  the  results  presented  in  the  earlier  report. 

Because  one  of  the  end  goals  of  this  project  is  to  be  able  to  identify  the  presence  of 
clouds  that  will  impact  transmittance  at  1.6  fm  in  the  Short  Wave  IR  (SWIR),  we  ran  an 
experiment  to  compare  the  cloud  imagery  from  the  WSI  with  that  from  a  fisheye  imager 
we  built  that  operates  at  1.6  pm  (Shields  et  al.  2003c).  Figure  50  shows  a  comparison  of 
the  visible  and  SWIR  during  the  day,  and  Figure  5 1  shows  a  comparison  near  sunset. 


Fig.  50.  Thin  clouds,  imagery  from  a  SWIR  system  at  1.6  pm  on  the  left,  and  the  WSI  in  the  visible  at  650 
nm  7  May  04  near  1230  Local 
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Fig.  51.  Sunset,  imagery  from  a  SWIR  system  at  1.6  |im  on  the  left,  and  the  WSI  in  the  visible  at  650  nm 
1 1  May  04  near  1942  L 

As  discussed  in  the  previous  report,  we  concluded  that  a  visible  system  will  detect  all  the 
clouds  that  impact  the  SWIR.  We  feel  that  the  visible  system  would  be  superior,  because 
it  has  much  better  sensitivity  and  can  work  at  night.  Even  in  the  daytime,  the  visible 
system  has  superior  imagery  due  to  low  noise  and  better  unifonnity  and  resolution. 

A  short  evaluation  of  Mid  Wave  IR  (MWIR)  system  characteristics  led  us  to  conclude 
that  use  of  a  MWIR  would  be  more  complicated  than  use  of  a  Long  Wave  IR  (LWIR) 
system,  and  would  have  other  disadvantages. 

We  did  an  extensive  evaluation  of  LWIR  systems,  evaluating  imagery  we  had  access  to, 
as  well  as  doing  theoretical  evaluations.  We  presented  results  for  zenith  angles  of  0°,  60°, 
and  85°.  As  an  example,  the  results  for  60°  are  shown  in  Table  3.  In  this  table,  we  show 
the  effective  temperature  of  the  cloud  signal,  for  low  (1  km),  mid  (5  km),  and  high  (10 
km)  clouds.  In  the  first  column,  the  entry  “10”  is  for  opaque  clouds  at  10  km,  and  “10 
thin”  is  for  thin  clouds  at  10  km.  “Aerosol”  indicates  the  calculations  for  the  aerosol 
background  at  the  same  look  angle.  These  calculations  are  derived  and  explained  in 
Shields  et  al.  2007.  The  effective  temperature  shown  for  each  altitude  has  been  corrected 
for  the  impacts  of  beam  transmittance  and  path  radiance,  as  explained  in  Shields  et  al. 
2007.  The  AT  columns  show  the  difference  between  the  effective  temperature  of  the 
clouds  and  the  effective  temperature  of  the  background  cloud-free  sky.  This  is  shown  for 
a  standard  atmosphere,  and  two  extremes:  the  winter  at  60°  N  latitude,  and  the  summer  at 
30°  N  latitude. 

In  Table  3,  the  color  pink  identifies  those  cases  where  the  temperature  difference  is  less 
than  %  of  the  estimated  31°  K  range  in  the  background.  The  table  entries  that  are 
identified  with  pink  are  cases  where  the  algorithm  would  have  to  be  reasonably 
sophisticated,  meaning  that  a  fixed  threshold  would  either  miss  the  middle  and  high 
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clouds,  or  would  identify  much  of  the  sky  near  the  horizon  as  cloud.  The  difference 
between  the  cloud  and  the  background  is  even  smaller  in  comparison  with  the 
background  variation  from  image  to  image.  The  need  to  take  into  account  seasonal, 
diurnal,  and  other  variations  in  the  background  signal  which  would  make  the  algorithm 
more  complicated.  The  blue  square  indicates  a  case  where  we  expect  that  the  cloud 
signal  will  be  lower  than  the  detection  threshold  of  the  instrument. 

Table  3 

Sample  IR  Evaluation  Results  from  Previous  Report 
Computed  Cloud  and  Background  Results  for  60°  Zenith  Angle 
(Colors  explained  in  Text) 


Standard  Atm 

Winter  60N 

Summer  3  ON 

Alt 

(km) 

Zen 

Eff 

Temp 

AT 

Eff 

Temp 

AT 

Eff 

Temp 

AT 

1 

60 

305.7 

22.7 

269.1 

22.0 

320.3 

29.3 

5 

290.1 

7.1 

254.3 

7.3 

299.3 

8.3 

10 

284.9 

1.9 

249.4 

2.4 

293.5 

2.5 

10  Thin 

284.0 

1.0 

248.2 

1.2 

292.2 

1.2 

Aerosol 

283.0 

247.0 

291.0 

There  are  some  LWIR  systems  in  development,  and  at  this  point,  the  imagery  we  have 
seen  do  not  appear  to  be  as  good  for  our  purposes  as  the  imagery  obtained  in  the  visible. 
A  theoretical  analysis  of  the  cloud  and  background  signals  indicates  that  although  low 
clouds  at  the  zenith  are  easy  to  detect,  sophisticated  algorithms  will  probably  be  required 
to  detect  low  clouds  at  angles  away  from  the  zenith.  Also,  a  reasonably  sophisticated 
algorithm  will  probably  be  required  to  detect  middle  and  high  clouds  at  most  angles. 
Close  to  the  horizon,  middle  and  high  clouds  are  expected  to  be  buried  in  the  noise.  In 
cold  environments,  the  middle  and  high  clouds  at  the  zenith  may  be  offscale  dark.  (The 
data  to  support  these  comments  are  shown  in  Shields  et  al  2007.) 

In  order  to  successfully  identify  the  presence  of  clouds,  two  things  are  necessary.  First, 
the  raw  imagery  must  have  sufficient  difference  between  the  cloud  and  background 
signals.  Second,  algorithms  must  be  able  to  sort  out  these  differences  and  identify  the 
presence  of  the  clouds.  Our  initial  analysis  leads  us  to  believe  that  for  an  IR  system,  the 
first  requirement  may  not  occur  for  many  needed  angles  and  cloud  heights,  and 
reasonably  sophisticated  algorithms  will  be  required  for  most  angles  and  cloud  heights. 
By  contrast,  as  illustrated  in  earlier  sections,  the  visible  sensors  provide  very  high  quality 


42 


imagery  under  all  conditions  we  have  encountered,  and  clouds  are  normally  well  detected 
down  to  the  horizon,  and  at  all  cloud  altitudes. 

Following  this  report,  the  sponsors  decided  that  although  they  may  want  us  to  develop  an 
IR  fisheye  system  for  test  at  a  later  time,  at  the  present  time  it  was  deemed  more 
important  to  continue  to  develop  the  capabilities  of  a  visible  system. 

9.  Hardware  and  Software  Developments  and  System  Preparation  for  Deployment 

As  part  of  this  contract,  we  evaluated  upgrades  that  should  be  done  prior  to  building  the 
next  instruments,  we  provided  maintenance  for  systems  in  the  field,  and  we  began 
refurbishment  of  older  WSI  units  for  use  in  the  SOR  deployments.  This  section  will 
provide  an  overview  of  the  system  evaluation,  the  upkeep  of  the  other  instruments  in  the 
field,  and  the  hardware  and  software  work  required  for  the  refurbishment. 

9.1.  Concepts  for  System  Upgrades 

In  designing  the  WSI  systems,  we  strive  to  achieve  a  system  that  is  very  capable,  reliable, 
and  as  cost-effective  as  possible  within  the  constraints  of  meeting  sponsor’s  technical 
needs  and  obtaining  the  required  capability  and  reliability.  The  Day/Night  WSI  systems 
were  first  designed  in  the  early  1990’s,  and  have  been  upgraded  in  several  respects  since 
that  time.  For  example,  the  solar/lunar  occultor  is  currently  much  more  reliable  than  the 
first  version  designed  in  the  early  1990’s.  Also  since  that  time,  the  environmental 
housings  have  been  upgraded  to  measure  and  report  on  the  state  of  the  instrument,  and 
respond  accordingly,  for  example  turning  the  camera  off  if  the  environmental  housing 
temperatures  are  too  high.  The  computer  and  electronics,  in  the  latest  versions,  were 
integrated  into  the  environmental  housing,  so  that  the  WSI  could  be  separated  from  the 
user  by  a  much  longer  distance,  and  be  connected  by  a  fiber  optic  communication. 

Several  upgrades  are  under  consideration  for  the  next  generation  of  instrument.  As 
discussed  in  Section  8,  we  feel  that  a  visible  system  will  provide  better  results  than  an  IR 
system  under  most  conditions.  However,  if  it  turns  out  that  we  are  unable  to  provide 
good  results  for  sunrise  and  sunset,  and  if  this  period  of  time  is  important  enough  to 
justify  the  incremental  cost,  we  could  evaluate  building  a  hybrid  system  that  takes 
advantage  of  both  a  visible  and  an  IR  sensor. 

One  of  the  major  changes  under  consideration  is  changing  to  a  smaller  and  higher 
precision  solar/lunar  occultor  that  does  not  block  so  much  of  the  sky.  As  reported  in 
Shields  et  al.  2007,  we  designed  a  solar/lunar  occultor  with  a  smaller  footprint  (i.e. 
smaller  obscured  area  in  the  sky),  but  we  were  not  happy  with  the  performance  of  the 
encoder  used  in  that  design.  Under  the  previous  contract,  we  tested  a  new  encoder,  and 
felt  this  would  perform  much  better. 

A  second  major  change  is  simplification  of  the  control  electronics.  The  current 
electronics  allow  the  user  to  control  each  of  the  peripherals  either  via  either  computer  or 
manual  control  on  the  Accessory  Control  Panel.  We  feel  this  flexibility  is  no  longer 
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needed,  and  plan  to  design  a  much  simpler  interface,  in  which  the  user  can  control  the 
components  independently  through  an  interactive  computer  interface.  One  possible 
computer  interface  design  is  shown  in  Figure  52. 


Fig.  52.  Initial  design  for  computer  control  to  replace  Accessory  Control  Panels 

With  modern  cameras,  the  same  or  better  cooling  of  the  CCD  chip  can  now  be  obtained 
with  air  cooling  of  the  camera.  This  would  eliminate  the  need  for  pumping  and 
monitoring  liquid  coolant  in  the  environmental  housing,  and  hopefully  enable  a  reduction 
in  the  size  of  the  housing  itself.  Additional  changes  are  under  consideration,  and  will  be 
more  fully  evaluated  if  we  are  funded  to  build  new  systems. 

9.2.  System  Maintenance 

During  this  period  of  this  contract,  there  were  several  repairs  to  Unit  12  at  Albuquerque. 
The  fdter  changer  photodiodes  failed  in  October  2004,  and  were  replaced  successfully  by 
site  personnel  with  our  support.  However,  they  failed  again  in  December,  and  the  MPL 
team  went  in  January  2005,  and  repaired  the  system,  as  documented  in  Memo  AV05- 
024t.  The  instrument  had  been  badly  damaged  by  a  nearby  lightning  strike  during  the 
previous  contract,  and  we  had  repaired  it  and  gotten  it  running  again.  In  April  2005, 
there  was  another  lightning  stonn  that  caused  odd  streaks  in  the  camera  image.  We  were 
not  able  to  repair  this  until  July  2005,  because  preparing  an  instrument  for  fielding  was 
higher  priority.  The  July  repair  trip  was  successful,  although  we  discovered  other 
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problems  that  were  not  show-stoppers.  This  trip  is  documented  in  Memo  AV05-027t. 
Later  in  July,  the  system  was  shut  down  during  a  lightning  storm,  and  in  the  process  of 
getting  it  restarted,  on-site  personnel  found  that  the  camera  cable  had  been  damaged  by 
rodents.  (It  operated  anyway  during  this  time.)  In  August  a  video  card  failed,  and  was 
sent  to  the  manufacturer  for  repair.  On  its  return  in  October,  it  still  failed  to  work.  The 
MPL  team  went  out  in  November,  and  made  a  number  of  repairs,  including  replacing  the 
cable  and  getting  the  video  to  work.  This  trip  is  documented  in  Memo  AV05-039t.  At 
that  point,  all  systems  were  in  good  repair,  except  the  air  conditioner.  The  air  conditioner 
should  be  replaced  when  feasible,  but  it  was  not  causing  problems  with  the  data,  even  in 
the  summer. 

There  were  also  a  couple  of  repairs  to  Unit  14  in  Virginia.  In  October  2004,  the  shutter 
failed.  Memo  AV04-044t  was  written  to  document  the  shutter  changing  procedures,  and 
a  shutter  was  sent  to  Virginia.  It  was  successfully  changed  by  the  site  personnel.  In 
March  2005,  a  new  computer  cooling  fan  was  sent  to  the  site  and  successfully  replaced 
by  site  personnel. 

The  shutter  failed  again  in  July,  and  this  time  site  personnel  were  not  able  to  get  it 
running.  A  repair  trip  was  made  by  the  MPL  team  in  August,  as  documented  in  memo 
AV05-026t. 

The  camera  failed  in  March  2006.  We  offered  to  swap  the  camera  with  another  one  here, 
but  this  was  not  acceptable  to  TASC  because  it  would  change  the  calibration. 
Unfortunately,  we  did  not  have  sufficient  funding  for  a  more  expensive  solution.  Under 
the  next  contract,  we  did  not  receive  sufficient  funds  for  a  full  time  hardware  person,  and 
SOR  felt  that  deployment  of  cameras  to  the  new  sites  was  a  higher  priority  with  the 
limited  resources. 

9.3.  WSI  Refurbishment  -  Hardware 

As  discussed  in  Section  5,  some  older  WSI  systems  became  available  early  in  the 
contract,  and  the  decision  was  made  to  refurbish  these  systems  for  use  at  three  new  sites 
for  this  program.  These  older  instruments  were  originally  built  for  the  Department  of 
Energy  (DOE)  and  used  for  many  years  at  a  variety  of  sites.  A  decision  was  made  in 
2004  to  retire  the  systems  (mostly,  I  believe,  because  the  algorithms  developed  by  the 
sponsors  were  not  adequate).  We  proposed  that  the  retired  instruments  be  given  to  MPL 
for  use  in  other  programs,  and  this  was  done.  As  a  result  of  this  situation,  our  SOR 
sponsors  asked  us  to  spend  more  of  the  Task  3  funds  on  refurbishment  of  these  systems, 
and  also  provided  the  optional  funding  in  the  contract  to  begin  to  adapt  three  of  the 
systems  so  they  could  be  fielded  at  sites  for  the  SOR  program. 

The  first  two  units  for  refurbishment,  Units  7  and  8,  were  received  in-house  in  February 
05,  as  documented  in  Technical  Memo  AV05-004t.  The  retired  unit  that  was  already  in- 
house,  Unit  4,  is  documented  in  Memo  AV05-020t.  Other  units  that  can  serve  as  test 
units  and  spare  parts,  or  be  used  for  other  programs,  are  documented  in  Memos  AV05- 
02 5 1,  AV05-028t,  and  AV06-022t. 
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The  configuration  of  the  Unit  7  sensor  and  the  controller,  upon  completion  of  the 
refurbishment,  are  shown  in  Figure  53.  The  left  side  shows  the  sensor  unit,  with  its 
environmental  housing  and  the  solar/lunar  occultor.  This  unit  has  successfully  operated 
for  many  years  in  the  Arctic,  and  similar  units  have  operated  in  other  locations,  including 
the  tropics  and  the  desert.  The  right  side  shows  the  controller  unit,  as  reconfigured  for 
the  SOR  project.  The  other  units  are  quite  similar,  except  for  the  size  of  the  occultor 
shade,  as  discussed  later  in  this  section.  Also,  Units  7  and  8  have  the  glass  domes,  and 
Unit  4  has  an  acrylic  dome. 


Fig.  53.  Sensor  and  Controller  Configuration  for  Unit  7 

These  systems  were  built  in  the  mid-to-late  1990’s,  and  as  a  result  they  needed  a  fair 
amount  of  refurbishment.  Typical  refurbishment  tasks  include  disassembly  and  cleaning, 
replacing  worn  components  such  as  the  coolant  tubing,  replacing  any  failed  components 
such  as  arc  drive  motors,  and  getting  the  cameras  tested  and  purged  at  Photometries. 
Unit  4  had  significant  electronics  issues  with  the  Accessory  Control  Panels  and  camera 
housings.  We  also  replaced  filters  as  necessary,  replaced  the  shutter,  replaced  cables  as 
required,  cleaned  the  system,  replaced  the  acrylic  optical  domes,  and  re-labeled 
connections.  These  repairs  were  completed  on  the  first  system  (Unit  7),  and  partially 
completed  on  two  other  systems  (Unit  4  and  8). 

The  solar/lunar  occultor  uses  an  arc  drive  to  provide  East-to-West  motion.  Normally,  the 
trolley  drive  moves  a  small  disk  along  a  North-South  axis,  in  order  to  obscure  the  sun  or 
moon  at  its  current  position.  Systems  7  and  8  used  an  occultor  shade  with  no  trolley,  and 
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the  trolley  drive  for  Unit  4  had  previously  been  cannibalized.  The  fixed  shade  is  a  piece 
of  metal  the  width  of  the  trolley,  but  it  covers  the  full  north-south  extent  of  the  solar  and 
lunar  motion,  so  that  it  doesn’t  need  to  move.  This  blocks  a  larger  part  of  the  sky  than  a 
trolley  would,  but  it  avoids  the  use  of  the  trolley  drive,  which  can  fail  particularly  in 
difficult  environments,  so  it  was  used  for  the  systems  fielded  in  locations  such  as  the 
Arctic.  The  fixed  shade  also  has  a  window  in  the  middle  to  enable  imaging  the  solar 
disk.  The  shade  size  depends  on  latitude,  and  is  thus  specific  to  the  location.  The  shade 
sizes  were  calculated  for  the  SOR  locations  and  rebuilt  accordingly.  The  occultor  shades 
are  documented  in  Memos  AV05-15t  and  -016t,  and  the  software  used  in  the  calculations 
is  documented  in  Memo  AV05-01  It.  Units  7  and  8  also  have  a  glass  dome  with  a  heater, 
in  place  of  the  more  standard  acrylic  dome. 

As  part  of  the  refurbishment  process,  the  systems  are  refocused  and  radiometrically 
calibrated.  The  electronics  such  as  the  meters  and  the  occultor  arc  drive  control  are 
recalibrated.  The  sensor  and  control  sub-systems  are  thoroughly  tested  prior  to 
deployment,  and  the  assembled  system  is  then  tested. 

These  older  systems  use  a  Photometries  Series  200  camera  system,  which  must  run  on  a 
DOS  computer  system.  Upgrading  the  cameras  to  a  Series  300  configuration  that  can  run 
on  more  modern  computers  was  considered,  but  the  sponsors  decided  it  would  be  too 
costly.  As  a  result,  we  kept  the  DOS  systems  for  the  WSI  control. 

Under  the  previous  DOE  program,  the  control  computers  had  used  a  split  backplane  with 
two  CPUs,  to  enable  a  networking  system  to  come  in  and  pull  data  files  without 
disturbing  the  data  acquisition.  Because  the  SOR  program  planned  to  use  a  better 
networking  system,  and  the  second  CPU  was  not  sufficiently  modem  to  use  for  the  real 
time  processing,  the  control  computer  was  reconfigured  to  disconnect  the  second  CPU 
and  its  associated  cards.  The  computers  were  also  refurbished  by  replacing  and/or 
repairing  any  cards  that  were  not  operational  on  arrival. 

A  WSI  Processing  computer  running  under  Windows  XP  was  added  to  enable  processing 
the  algorithm  in  real  time.  The  appropriate  networking  cards  were  added  to  the 
processing  computer.  A  Masterview  switch  was  used,  to  allow  the  use  of  either  computer 
with  the  same  keyboard,  mouse,  and  monitor.  An  updated  GPS  was  added  because  the 
older  one  was  no  longer  able  to  get  a  signal,  and  Gannin  no  longer  supported  that  model. 

Also,  a  remote  power  switch  was  installed  to  monitor  the  status  of  the  control  computer 
via  the  processing  computer.  Software  was  written  so  that  if  the  processing  computer 
stops  receiving  data  from  the  control  computer,  the  processing  computer  will  reboot  the 
control  computer  via  this  remote  power  switch.  Then  the  control  computer  will 
automatically  restart  the  data  acquisition  program  upon  reboot  and  initiate  a  connection 
with  the  processing  computer.  (The  WSI  is  also  installed  in  such  a  way  that  if  the 
processing  computer  hangs,  the  personnel  at  SOR  can  detect  this  and  reboot  it.) 

We  decided  to  house  the  WSI  controller  in  the  small  rack-mount  shown  in  Fig.  53,  which 
is  shippable  without  an  external  crate.  We  mounted  the  processing  computer  on  top  of 
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the  rack  mount.  In  this  way,  the  controller  has  a  smaller  profile  than  when  it  is  mounted 
in  the  more  standard  full  rack  mount.  This  was  done  primarily  because  the  controller 
rack  will  be  placed  in  a  small  and  very  crowded  shed,  and  it  minimizes  the  required 
space,  as  well  as  minimizing  the  heat  created  by  the  controller.  With  the  more  standard 
enclosed  rack  we  use,  the  rack  itself  requires  a  cooler  if  the  environment  temperature 
control  is  marginal,  as  it  may  be  in  the  sheds.  With  the  smaller  and  more  open  rack 
configuration  used  for  this  fielding,  no  additional  cooler  is  required,  although  it  is  still 
necessary  that  the  shed  be  cooled. 

Following  receipt  of  the  first  unit  in  February  2005,  we  began  working  intensively  on  the 
system  refurbishment,  because  the  sponsors  were  hoping  to  deploy  the  first  one  soon. 
Due  to  peculiarities  in  the  funding  process,  the  processing  computers  were  provided 
directly  by  the  sponsor,  and  were  received  in  late  April  or  early  May  of  2005.  Following 
the  installation  of  the  processing  computer  into  Unit  7,  Unit  7  was  tested  outside 
successfully  starting  May  12.  At  that  time  we  had  not  yet  calibrated  the  system,  nor  done 
miscellaneous  clean-up  such  as  labeling  cables.  This  was  done,  along  with  numerous 
tests,  in  May  and  June.  The  full  automated  hands-off  test  began  June  27,  although  the 
new  occultor  shade  was  added  after  this  date. 

As  it  turns  out,  our  sponsors  had  to  request  that  we  delay  the  deployment  due  to  issues 
outside  of  the  control  of  MPL.  As  a  result,  Unit  7  was  not  actually  deployed  until  1  May 
2006,  under  the  next  contract.  However,  during  this  time,  Unit  7  was  allowed  to  run 
continuously,  and  ran  well.  We  ran  the  control  computer  and  WSI  system  in  hands-off 
mode,  but  we  also  frequently  stopped  the  processing  computer  in  order  to  test  ongoing 
updates  to  the  processing  code  that  will  be  discussed  in  the  next  section.  The  memo 
AV06-007t  documenting  the  hardware  was  written  later,  but  is  also  referenced  here. 

During  this  time,  we  worked  on  repair  of  the  systems  in  the  field,  as  documented  in 
Section  9.2,  and  worked  on  refurbishment  of  Units  4  and  8,  which  are  the  other  two  units 
for  deployment.  By  October  2005,  Unit  8  was  running  indoors,  although  it  still  needed  a 
significant  amount  of  work,  such  as  calibration  of  the  electronics.  By  the  end  of  the 
contract  in  April  2006,  Unit  8  was  running  outdoors. 

In  summary,  Unit  7  was  completely  refurbished  and  ready  for  deployment  when  the 
sponsors  were  ready.  It  ran  very  well  through  the  extensive  test  period.  The  other  two 
units  were  partially  refurbished,  and  this  work  was  continued  under  the  next  contract. 
Because  the  sponsors  were  not  ready  for  deployment  under  this  contract,  the  deployment 
was  delayed  until  the  follow-on  contracts. 

9.4.  WSI  Refurbishment  -  Software 

Along  with  the  hardware  changes,  significant  software  changes  were  required  to  the 
systems  under  refurbishment  in  order  to  enable  fielding  them  for  the  SOR  deployments. 
Some  of  these  changes  were  due  to  the  differing  needs  of  the  older  project  and  the  SOR 
project,  and  some  were  due  to  the  hardware  upgrades.  Once  these  initial  changes  were 
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made,  significant  effort  was  put  into  making  the  programs  more  robust  and  convenient 
for  the  remote  user. 

There  are  two  primary  programs  on  the  system:  the  WSI  control  program,  RunWSI,  and 
the  program  that  processes  the  algorithm  on  the  processing  computer,  ProcWSID.  The 
control  program  was  updated  to  change  the  acquisition  interval.  The  previous  DOE 
project  required  acquisition  of  a  full  data  set  every  6  minutes,  and  red  or  open  hole  every 
2  minutes.  The  SOR  task  required  a  full  image  set  every  1  minute  during  the  daytime, 
and  2  minutes  at  night.  The  program  was  changed  so  that  the  acquisition  interval  is  user- 
selected,  and  it  is  currently  set  for  1  minute  intervals  during  the  day  and  2  minutes 
intervals  at  night.  The  program  was  updated  so  that  it  no  longer  has  the  option  to  acquire 
either  red  or  open  hole  imagery  at  night;  it  only  acquires  open  hole,  except  under  full 
moon.  During  the  full  moon  night,  the  system  acquires  both  open  hole  and  spectral  data 
in  all  filters,  so  that  we  can  evaluate  which  are  best  for  the  algorithms.  Once  this  decision 
is  made,  we  anticipate  acquiring  either  open  hole  or  spectral  data,  but  not  both  under  full 
moon. 

The  control  program  was  modified  to  communicate  with  the  processing  computer  via  ftp. 
Several  features  of  the  earlier  program  that  are  not  in  the  other  SOR  programs  were 
retained.  These  include  giving  priority  to  grabbing  images,  even  if  the  occultor  is  not 
quite  in  the  right  place  or  the  filter  is  not  yet  in  position,  and  delaying  a  minute  if  the  0- 
second  mark  is  missed  so  that  image  sets  always  start  on  the  0  mark. 

The  image  archive  format  was  changed  to  be  compatible  with  the  other  SOR  programs. 
The  user  hot  key  option  was  removed,  because  it  interfered  with  the  timing  at  these  short 
acquisition  intervals.  The  ability  to  either  overwrite  or  append  to  diagnostic  files  was 
added. 

Various  system  diagnostics  were  added.  For  example,  timing  information  related  to  the 
length  of  the  image  grabs  was  added  to  the  header,  to  enable  diagnosing  problems  related 
to  hardware.  A  feature  that  had  been  in  some  versions,  but  not  in  the  SOR  systems,  is  a 
check  to  detennine  if  the  shutter  is  not  opening  properly.  These  diagnostics  and  others 
are  documented  in  Memos  AV05-023t  and  AV06-005.  In  addition,  we  added 
information  to  the  headers,  to  be  used  by  the  processing  program  in  creating  the  QC  files, 
including  the  user-defined  night  acquisition  interval  and  the  time  source.  The  code  was 
speeded  up,  to  enable  reliable  1 -minute  data  acquisition.  Another  change  involved 
updating  the  program  to  accept  the  new  GPS  output  strings. 

The  new  processing  computers  for  Units  4,  7,  and  8  run  under  WindowsXP.  Previously, 
these  WSI  units  did  not  include  real-time  algorithm  processing.  As  part  of  this 
refurbishment,  we  took  the  processing  program  from  Units  13  and  14,  designed  to  run 
under  WindowsNT,  and  updated  it  for  this  environment.  This  program  included  the  day 
cloud  algorithm.  Units  13  and  14  also  include  the  contrast-based  night  algorithm; 
however  it  is  called  separately  as  a  system  call. 
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A  couple  of  reboot  options  were  added  to  ProcWSID.  1)  Processing  computer  reboot  - 
To  free  memory  and  refresh  the  operating  system  an  option  was  added  to  reboot  the 
processing  computer  at  any  user  specified  time  and  period.  For  example,  Unit  7  is  setup 
to  reboot  every  day  at  1000Z  (added  24  Oct  2005).  2)  Control  computer  reboot  -  if  no 
data  is  received  for  any  user  specified  period  the  processing  computer  can  reboot  the 
control  computer.  For  example,  the  user  can  specify  that  if  no  data  is  received  from  the 
control  computer  for  30  minutes,  the  program  can  be  set  to  either  note  the  event  in  the 
reboot  log  (C:\Program  Files\ProcWSI\WSILiveCheck.txt)  or  to  reboot  the  computer 
(added  1 1  May  2005). 

One  of  the  major  improvements  to  the  processing  program  was  the  development  of  a  new 
user  interface,  shown  in  Figure  54.  This  user  interface  shows  the  status  of  each  of  the 
peripheral  systems  such  as  environmental  housing  temperature.  It  shows  a  green,  yellow, 
or  red  light  beside  each  peripheral  subsystem  depending  on  the  status  of  the  system. 
Technical  Memo  AV05-022t  was  written  for  the  support  personnel  in  the  field,  to  explain 
how  to  check  the  WSI  system. 


■  ProcWSID 

F!e  Help 

Red  Image 

Date:  07/10/2005 
Time:19:47G 

Exposure:  100 

■  Arc  Actual:  80.1 
Arc  Dest:  80.3 

■  Trie  Actual:Fixed 
Trie  Dest:  Fixed 

•  ND:  3  (3  log) 

■  SP:  3  (Red) 

<  CCD  chip  temp:  -34.0 
i  Env.  Housing  temp:1 6.0 
i  Cam.  Housing  tcmp:24.0 
Flow:  0.38  GPM 

■  N2:  0.0  PSI 

■  Flux  Level  Status 

'  WSI  Hard  Disk  Status 

Min.:  768 
Max.:  12600 


[Next  grab  time:  19H8Z 


Figure  54.  New  User  Interface,  to  enable  inexperienced  users  to  assess  system  status 

Other  features  that  were  added  to  ProcWSID,  some  prior  to  June  05  and  some  later  in  the 
year,  include  the  following: 
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1)  All  I/O  paths  are  specified.  Various  input  files  are  used  for  the  occultor  mask,  day 
and  night  algorithms.  Rather  than  determining  which  input  file  to  use  based  on 
header  information,  we  state  which  input  files  to  use  in  ProcWSID.inp.  Also, 
different  output  paths  are  specified  for  the  different  archives. 

2)  All  I/O  paths  are  verified  during  initialization  -  The  program  alerts  the  user  early  in 
the  program  if  there  are  any  missing  input  files.  Also,  the  program  creates  the  output 
directories  if  they  don’t  exist. 

3)  The  occultor  mask  routine  was  updated  to  handle  the  fixed  occultor  shade. 

4)  There  were  some  features  in  the  Unit  13/14  code  having  to  do  with  another  system 
that  is  no  longer  in  use.  Since  these  features  were  no  longer  needed,  they  have  been 
commented  out. 

5)  Files  are  converted  to  a  long  file  fonnat  -  This  was  done  to  make  it  easier  for  SOR  to 
identify  WSI  data  on  their  systems.  The  new  filename  format  is 
WSIuuummddyyyyhhmm.nnn;  where  uuu  is  unit  number  mmddyyyyhhmm  is  the 
date  and  time  of  the  image  grab  and  nnn  is  the  image  type. 

6)  QC  files  have  been  added  for  the  different  WSI  components.  See  Tech.  Memo 
AV05-023t  for  information  on  the  QC  files. 

7)  An  option  to  write  to  external  300  GB  disk  was  added.  Archiving  to  an  external  disk 
provides  a  backup  in  case  of  ftp  failure,  and  also  makes  transfer  of  a  copy  of  the  data 
to  MPL  easier.  These  disks  need  to  be  replaced  approximately  every  6  months. 

8)  A  12-character  error  string  is  written  to  headers  of  the  ratio  and  cloud  decision  images 
to  let  the  user  know  about  any  problems  with  the  imager  that  would  affect  cloud 
decision  results.  The  error  string  details  are  documented  in  Memo  AV06-006t. 

9)  The  program  sorts  the  files  before  processing  the  algorithm.  This  was  done  so  that  if 
there  is  a  backlog  of  files  to  process,  the  files  will  be  processed  in  order  by  date  and 
time,  with  the  earliest  files  processed  first. 

During  May  and  June  05,  the  code  was  tested  extensively,  particularly  in  failure  mode. 
For  example,  we  tested  how  the  software  handles  the  situation  if  the  Accesssory  Control 
Panel  (ACP)  is  accidentally  set  in  “local”  rather  than  “computer”  mode.  We  also  turned 
off  the  ftp  server  program  on  the  processing  computer  (that  communicates  with  the 
control  computer)  to  verify  that  RunWSI  still  runs  if  it  can’t  ftp  data  and  ProcWSID 
reboots  the  control  computer  after  it  doesn’t  receive  data  after  30  minutes.  We  also  tested 
the  program  under  full  moon  conditions  and  we  turned  off  the  CEU  to  verify  that 
RunWSI  properly  flags  the  case  where  we’re  not  receiving  a  camera  signal.  Following 
these  tests,  we  started  the  Unit  7  control  computer  running  in  hands-off  mode  on  June  27. 
The  RunWSI  program  ran  without  failures,  and  it  was  run  essentially  continuously  until 
we  began  final  preparations  for  the  deployment  in  May  2006  after  the  new  funding  was 
received.  Similarly,  the  hardware  performed  flawlessly  during  this  period  except  for 
occasional  but  rare  spectral  filter  wheel  skips. 

In  the  meantime,  we  continued  to  make  extensive  upgrades  and  tests  of  the  processing 
program,  while  running  the  control  program.  While  testing  the  processing  program  we 
found  that  we  were  having  memory  leak  problems.  We  traced  the  problem  to  running  the 
night  algorithm  as  an  independent  program  under  DOS.  So  we  decided  to  incorporate  the 
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night  algorithm  into  the  ProcWSID  program.  This  was  an  extensive  job,  but  well  worth 
the  effort,  and  it  solved  the  memory  leak  problem. 

Although  originally  the  programs  to  ftp  data  from  the  site  to  SOR  were  to  be  written  by 
another  sponsor,  this  had  not  been  done  yet,  so  we  wrote  and  tested  several  versions  of 
the  ftp  programs.  Other  features  were  added.  For  example,  we  added  much  more 
sophisticated  data  QC  files  to  the  output  of  the  processing  program.  The  first  version  of 
these  QC  files  is  documented  in  Memo  AV05-023t.  In  September  2005,  the  SOR  team 
wrote  a  program  to  automatically  evaluate  these  QC  files  and  identify  cases  that  require 
attention.  In  order  to  avoid  confusion  between  units,  ProcWSI  was  adapted  to  add  the 
WSI  unit  number  to  both  the  raw  and  the  processed  filenames.  A  program 
ProcWSIDInput  was  written,  to  allow  the  user  to  update  the  program  inputs  in  a  safer 
way,  and  the  first  version  was  installed  in  March  2006. 

A  considerable  amount  of  supporting  software  work  was  also  done  under  this  contract. 
Some  of  it  is  mentioned  in  Sections  6  and  7  that  discuss  the  algorithms.  Along  with  the 
instruments,  we  received  custody  of  a  field  calibration  device,  designed  for  easy  and 
convenient  radiometric  calibrations  of  the  WSI.  The  field  cal  acquisition  software  was 
updated  to  provide  the  flexibility  needed  for  the  SOR  program.  In  addition,  a  second 
version  of  the  Fieldcal  acquisition  software  was  created  to  operate  under  the  WindowsNT 
operating  system  used  on  Units  13  and  14.  WSITest,  a  program  designed  to  enable 
convenient  test  of  the  WSI  hardware  components,  had  been  written  for  Units  13  and  14, 
and  was  adapted  to  work  with  Unit  12,  and  installed  there  in  July  2005.  (There  is  similar 
code  that  works  for  the  DOS  Units  4,  7,  and  8.)  Program  WSIView  was  written  to  allow 
a  user  in  the  field  to  evaluate  the  imagery. 

Other  support  programs  include  several  programs  written  to  test  WSI  system  components 
such  as  the  remote  power  switch,  and  several  programs  written  to  test  various  versions  of 
the  ftp  logic.  Several  other  support  programs  to  help  with  data  analysis  were  written. 
These  include  image  looping  programs  (one  for  Windows,  and  several  to  operate  in  an 
image  processing  V++  environment),  a  variety  of  programs  written  to  enable  easier 
handling  and  sorting  of  these  large  data  bases,  and  programs  for  easier  evaluation  of  the 
results  of  algorithm  processing. 

As  discussed  in  Section  9.3,  Units  7  and  4  were  fielded  under  the  next  contract,  which 
will  be  documented  in  a  later  report.  Prior  to  these  deployments,  the  upgrades  which  had 
been  tested  on  Unit  7  were  also  installed  and  tested  on  Unit  4.  Technical  Memo  AV06- 
002t  discusses  the  directories  and  files  on  the  control  computer.  Memos  AV06-003t  and 
-004t  provide  an  operations  overview  for  the  control  computer  and  the  processing 
computer,  respectively.  Memos  AV06-005t  and  -006t  more  fully  document  the  RunWSI 
and  ProcWSID  software. 

In  the  future,  in  addition  to  continuing  research  on  algorithm  upgrades  and  evaluation  and 
supporting  programs,  we  also  need  to  update  the  data  acquisition  and  processing  code  for 
the  SOR  Unit  12  so  that  it  will  have  these  new  features.  The  Unit  12  control  code  runs 
under  Windows95,  so  it  will  require  significant  changes  to  provide  the  upgrades  inherent 
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in  the  DOS  control  code  used  on  Units  4,  7  and  8.  Installing  the  new  processing  code 
should  be  relatively  straight-forward.  A  new  version  was  installed  in  November  2005, 
but  it  does  not  include  all  the  current  features. 

9.5.  Deployment  Logistics  Support 

The  three  sites  are  being  prepared  by  other  contractors  for  SOR.  As  part  of  the  work  on 
the  hardware  and  software,  we  had  to  coordinate  numerous  details,  such  as  how  much 
power  is  needed,  and  how  many  cables,  the  physical  dimensions  of  the  WSI  installation, 
the  size  of  shipping  crates,  and  so  on.  These  details  were  mostly  worked  out  in  early 
2005. 

In  addition,  we  do  quite  a  bit  of  documentation  prior  to  and  just  after  deployments.  Most 
of  this  was  done  under  the  next  contract,  since  the  site  was  finalized  for  deployment 
during  the  period  of  the  next  contract.  However,  the  software  documentation  memos 
AV06-002  through  006  were  written  under  this  contract.  Also,  the  initial  version  of  the 
Units  7  and  8  setup  memos,  Memos  AV05-030t  and  AV06-001t,  were  written  under  this 
contract.  The  setup  memos  were  originally  written  for  the  DOE  program,  because  the 
program  initially  required  that  the  instruments  be  set  up  by  untrained  personnel. 
Although  the  level  of  detail  included  in  these  memos  is  beyond  what  is  required  by  either 
SOR  or  MPL  personnel,  we  find  that  much  of  the  information  in  the  memos  can  be 
useful,  so  we  have  continued  providing  them.  Memo  AV06-007t  documents  the  general 
instrument  overview  for  Unit  7. 

10.  Summary 

Under  this  contract,  we  had  the  opportunity  to  make  significant  upgrades  to  algorithms 
and  our  state  of  knowledge  of  the  algorithm  results.  In  particular,  we  upgraded  the  day 
algorithm  to  use  the  NIR  data  at  hazy  sites,  and  we  wrote  programs  to  assess  the  accuracy 
of  the  results.  We  developed  concepts  for  ground-truthing  the  night  algorithm 
automatically,  and  further  developed  concepts  for  the  high  resolution  night  algorithm. 
We  processed  and  evaluated  a  test  bed  data  set  of  both  day  and  night  data,  and  were  able 
to  assess  the  strengths  and  weaknesses  of  the  algorithms.  We  made  significant  progress 
toward  a  transmittance-based  night  cloud  algorithm.  A  considerable  amount  of  support 
software  for  use  in  algorithm  processing  and  development  was  written. 

Infrared  systems  were  evaluated,  and  found  to  have  significant  problems  for  our 
application.  The  decision  was  made  to  postpone  further  evaluation  of  these  systems. 

Potential  WSI  system  upgrades  were  evaluated.  Older  WSI  systems  became  available  to 
MPL  during  this  program,  and  the  decision  was  made  to  begin  refurbishing  them  for  use 
at  SOR  deployments.  One  unit  was  refurbished  successfully,  with  very  significant 
updates  to  the  software,  and  two  units  were  partly  refurbished.  The  software  was  updated 
not  only  to  enable  real  time  algorithm  processing  on  these  older  systems,  but  the  night 
algorithm  was  integrated  into  the  actual  processing  code  for  the  first  time.  The  code  was 
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adapted  in  several  ways  for  current  program  needs,  especially  including  features  to  make 
it  more  robust  and  features  to  enable  more  efficient  QC  of  the  data. 

We  believe  that  we  have  completed  all  contract  requirements.  We  very  much  appreciate 
having  had  the  opportunity  to  do  this  work. 
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